Treasury Board of Canada Secretariat
Symbol of the Government of Canada

ARCHIVED - Mid-Term Evaluation of the Implementation of the Cabinet Directive on Streamlining Regulation


Warning This page has been archived.

Archived Content

Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.

3. Conclusions and Supporting Findings

3.1 Relevance

3.1.1 Do the Objectives of the Cabinet Directive on Streamlining Regulation Align With Federal Government Priorities and the Treasury Board of Canada Secretariat's Strategic Outcome?

Conclusion: The objectives of the CDSR do align with federal government priorities of protecting and advancing the public interest and competitiveness through a more effective, efficient and accountable regulatory system. They also align with the Treasury Board of Canada Secretariat's strategic outcome.

The CDSR objectives are to:

  • protect and advance the public interest;
  • promote a fair and competitive market economy;
  • make decisions based on evidence;
  • create accessible, understandable and responsive regulation;
  • advance the efficiency and effectiveness of regulation; and
  • require timeliness, policy coherence and minimal duplication.

The CDSR is the mechanism by which the government achieves its regulatory priorities. Since its implementation in April 2007, every Speech from the Throne and Budget Speech has included references to regulatory priorities that are not only consistent with CDSR objectives, but that are also expected to be more easily attained because of the CDSR. For example, the 2010 Speech from the Throne stated: “To support responsible development of Canada's energy and mineral resources, our Government will untangle the daunting maze of regulations that needlessly complicates project approvals, replacing it with simpler, clearer processes that offer improved environmental protection and greater certainty to industry.”

3.1.2 Is There a Continued Need for the Cabinet Directive on Streamlining Regulation? Do the Principles of the CDSR Respond to the Identified Need for Improvement in Regulations?

Conclusion: The CDSR responds to the needed improvements to Canada's regulatory approach identified by the External Advisory Committee on Smart Regulation.

The Government of Canada amends or implements approximately 250 regulations a year. Of these, approximately 10% are considered high-impact: they may be highly controversial; they may represent a significant change to the status quo; or they may seriously affect such areas as health and safety, the environment, the economy or government. Given the number and complexity of regulations, a policy or framework is needed to guide the process of implementing regulations efficiently and effectively to support the well-being of Canadians. The CDSR supports the government's commitment to protect and advance the public interest by working with Canadians and other governments so that regulatory activities provide the greatest possible benefit to current and future generations of Canadians.

The External Advisory Committee on Smart Regulation was created to provide the federal government with external perspective and expert advice on how to redesign its regulatory approach for the twenty-first century. Every major industrial sector told the Committee that the existing regulatory system often acted as a constraint to innovation, competitiveness, investment and trade. The Committee's report noted that if the regulatory “system is not aligned with new developments and practices of the twenty-first century, it may put Canadians' safety at risk and affect citizens' trust in government.” Furthermore, “other countries are reforming their systems, and Canada cannot afford to be left behind.”[4]The CDSR responds to both the recommendations of the Committee and the needs of the regulatory system.

3.2 Effectiveness

3.2.1 Is the Cabinet Directive on Streamlining Regulation on Track Toward Achieving All of Its Intended Outputs? (capacity to implement the CDSR)

Conclusion:The implementation of the CDSR is on track to produce most of its intended outputs. However, some gaps, such as the development of service standards, need to be addressed in the next two and a half years.

The administrative data show that the following outputs have been achieved:

  • Training—Five regulatory courses have been developed, and each has been delivered on several occasions to federal regulators through the Canada School of Public Service.
  • Partnership development—Strategic partnerships have been established with academic organizations, departmental and agency regulatory units, and other jurisdictions.
  • Provision of advice and expertise through theCentre of Regulatory Expertise—Advice has been provided to federal departments and agencies on issues relating to cost-benefit analysis, performance measurement, and Regulatory Impact Analysis Statements.
  • Research and information sharingTBS-RAS has published reports and delivered presentations and workshops.
  • Developing and refining policyTBS-RAS has developed tools to assist regulators in areas such as cost-benefit analysis, risk assessment, and performance measurement and evaluation.
  • Regulatory coordination functionshave been established in most departments.

The key gaps in output that TBS-RAS needs to address in the next two and a half years are (a) the development of service standards for regulations (as part of its policy development and refinement activities) and (b) linking departments and agencies to external experts (as part of its assistance to departments' activities). In addition, TBS-RAS compliance reports on the results of TBS-RAS challenge activities have been delayed.

Under the CDSR, “Departments and agencies are responsible for putting in place the processes to implement regulatory programs and to manage human and financial resources effectively, including publishing service standards, including timelines for approval processes set out in regulations, setting transparent program objectives, and identifying requirements for approval processes.” TBS-RAS had intended to provide guidance on service standards to assist organizations in complying with this requirement, but this guidance has been delayed to ensure consistency with the new Treasury Board Policy on Service Standards, which has yet to be finalized.

While somewhat delayed, TBS-RAS has started to link departments and agencies with external experts to enable them to meet CDSR requirements using Regulatory Cooperation Plans (RCPs) or specific procurement streams. RCPs are a funding mechanism that helps departments increase their capacity to meet CDSR requirements. At the time of the evaluation, RCPs were in place for seven departments having a high volume of regulations. However, they were not in place for other departments having low capacity in at least one of the CDSR requirement areas. In fact, a number of departments surveyed that had obtained funding under RCPs were unaware that they had received funding. This perceived lack of funding does not affect the progress toward outputs, but it suggests that more focused communications in this area by may be required.

TBS-RAS activities also include the analysis and challenge of departmental regulatory proposals. The output of this activity was meant to be compliance reports shared with the regulatory community and used to identify gaps that needed to be addressed. TBS-RAS is now focusing on producing these performance reports and has completed them for the first two quarters of fiscal year 2009–10. Plans have also been made to complete performance reports going back to 2004, which will allow for a comparison of regulatory proposals pre- and post-CDSR.

3.2.2 To What Extent Are the Treasury Board of Canada Secretariat's Regulatory Affairs Sector andDepartments and Agencies Satisfied With the Outputs Produced? Are the Outputs Produced by TBS-RAS Responding to the Capacity Needs of Departments and Agencies?

Conclusion: Generally, departments and agencies were quite satisfied with TBS-RAS products and services used, but less satisfied with the level of TBS-RAS communications and partnering. In addition, some departments and agencies are not aware of (or are not using) the full range of TBS-RAS outputs and services, including partnering, a finding that is consistent with insufficient levels of communication.

TBS-RAS staff reported that they were satisfied with the outputs produced. Some areas for improvement were identified, but there was no consistent theme. Just over 50% of departmental survey respondents reported that they were generally satisfied with the outputs produced by TBS-RAS, including Canada School of Public Service courses, advice on how to meet CDSR requirements, and analysis of proposals.[5]

However, at least 30% of survey respondents reported that they were not satisfied with efforts of TBS-RAS to foster partnerships or with the overall communications strategy for the implementation of the CDSR. In addition, just over 50% of survey respondents could not comment on some of the outputs produced by TBS-RAS (e.g., funding agreements or partnerships), suggesting a possible lack of awareness or use of these services. These survey findings (lower levels of satisfaction with partnerships and communications, and an inability to comment on some outputs) applied to departments and agencies having both low and high regulatory activity.

  • When asked about the level of satisfaction with partnerships developed within government and with other jurisdictions, those survey respondents who provided a rating indicated that they were less satisfied with work done in this area, but many provided a response of “don't know.” The program data reveal that partnerships have been developed internationally, with provinces and territories, with universities, within the federal government, and with the Community of Federal Regulators, indicating that TBS-RAS's Policy and Centre of Regulatory Expertise divisions are active in developing partnerships. The apparent disconnect between the survey responses and the document review suggests that survey respondents who indicated a lower level of satisfaction may have done so because of a lack of knowledge about TBS-RAS efforts rather than because of dissatisfaction with partnering outputs. It should be noted that while survey responses of “don't know” were particularly high for questions relating to partnering, they were also as high as 50% for some other areas, suggesting a general lack of communication by TBS-RAS to departments and agencies about the specific services offered.

3.2.3 Has the Treasury Board of Canada Secretariat's Regulatory Affairs Sector Experienced Any Barriers to Producing the Desired Level of Outputs?

Conclusion: The most significant barrier to producing CDSR outputs, both in TBS-RAS and in departments and agencies, is the recruitment and retention of qualified personnel in specific areas of expertise (e.g., cost-benefit analysis and Performance Measurement and Evaluation Plans).

TBS-RAS has not identified any barriers to producing outputs, with the exception of finding and keeping qualified personnel. Some of the department interviewees did report that TBS-RAS did not have enough resources or experienced staff, especially given issues of employee turnover.

When interviewees were asked about the sustainability of the changes in departmental capacity to respond to the CDSR, 25% expressed concerns about employee turnover and recruitment. To help alleviate resource pressures, TBS-RAS and the Community of Federal Regulators have been working together to create a pool of qualified regulators who would be available to all departments and agencies to help address gaps in capacity.

3.2.4 To What Extent Is the Treasury Board of Canada Secretariat's Regulatory Affairs Sector on Track Toward Achieving Expected Outcomes Within the Next Two and a Half Years? To What Extent Has There Been an Increase in Capacity by Departments and Agencies to Meet CDSR Requirements?

Conclusion: TBS-RAS has completed a large part of the foundation work to bring about the change in expected outcomes within the next two and a half years. Departments and agencies have achieved increased capacity in all targeted areas―coordination and cooperation, regulatory service standards and assessment of administrative requirements, cost-benefit analysis, and Performance Measurement and Evaluation Plans.

The CDSR immediate outcomes are intended to increase the following capacities of departments and agencies to meet CDSR requirements, while tailoring effort to the level of risk associated with regulatory proposals:

  • Capacity for departments and agencies to coordinate and cooperate with each other and across jurisdictions
  • Capacity for departments and agencies to use and communicate regulatory service standards and to assess administrative requirements
  • Capacity for departments and agencies to carry out cost-benefit analyses of regulations
  • Capacity for departments and agencies to design and implement Performance Measurement and Evaluation Plans

Although it was not expected that CDSR outcomes would be fully achieved by the mid-term of the implementation period, the evidence shows that, overall, capacity in the above areas is increasing. Departmental self-assessments of capacity to meet CDSR requirements, and of compliance with those requirements, show that outcomes are beginning to be achieved.

Survey respondents were asked to retrospectively rate their capacity to deliver on immediate outcomes in April 2007, when the CDSR was introduced, and again in fall 2009. The results are shown in Table 2. The average capacity rating in 2007 was highest (an average of 3.45 out of 5) for coordination and cooperation, and organizations rated this capacity as only slightly (10%) higher in 2009. Organizations rated themselves as having achieved significant increases, ranging from 23% to 72%, in the other four capacity areas. Because of the moderate ratings for all five capacity areas, however, there continues to be opportunity for improvement. It should be noted that the capacity to design and implement Performance Measurement and Evaluation Plans for regulations was rated the lowest, with an average rating of 2.62. This low rating may be due to the relatively new requirements to develop Performance Measurement and Evaluation Plans for high-risk regulations, which were introduced only in July 2009. The greatest changes in capacity were identified in those areas where organizations initially rated the weakest, indicating that investment in these areas was well targeted.

Table 2. Average Rating of Department Survey Respondents of Their Capacity in April 2007 Versus Fall 2009a
  Capacity Rating No Answer Change in Rating Increase (%)
April 2007 Fall 2009
a. Survey respondents rated their capacity using a five-point scale, where 1 was no capacity and 5 was fully capable.
Coordinate and cooperate with other federal departments and agencies and across jurisdictions in the development of regulations 3.45 3.80 6 0.35 10
Assess the administrative requirements of regulations with a view to making them less burdensome 2.94 3.63 6 0.69 23
Use and communicate regulatory service standards 2.75 3.49 9 0.74 26
Carry out cost-benefit analysis of regulations 2.33 3.04 4 0.71 37
Design and implement Performance Measurement and Evaluation Plans for regulations 1.52 2.62 5 1.10 72

In terms of support for capacity building, just over 50% of departments and agencies reported that the support provided by TBS-RAS was adequate. On average, organizations attributed nearly half of the change in their capacity to TBS-RAS. Perhaps not surprisingly, areas in which organizations rated their capacity as lowest also received the lowest ratings of satisfaction with TBS-RAS services.

One indicator of the capacity of departments and agencies to comply with CDSR requirements is the extent to which they do comply with those requirements. Although organizations rated their capacity in the mid-range, the performance reports for the first two quarters of 2009–10 actually show a high level of compliance (90%) with CDSR requirements. The only area below 90% is the capacity for cost-benefit analysis, which is still high, at 86% for the first quarter and 73% for the second quarter. Even with high compliance rates, respondents from departments and agencies for both the survey and interviews identified a need for more feedback from TBS-RAS on their organization's progress in meeting CDSR requirements.

3.2.5 Were There Any Unintended Impacts of the Implementation of the Cabinet Directive on Streamlining Regulation?

Conclusion: Unintended impacts reported by survey respondents include increased workload and time requirements combined with resource levels that have not kept up with the increased workload.

Among survey respondents who reported unintended impacts, approximately 25% saw increased workload and time requirements for the development of Regulatory Impact Analysis Statements and triage statements, as well as increased analysis requirements for high-impact regulations. A few respondents reported that their departmental regulatory function had not kept up with the increased requirements and that it lacked resources to meet the requirements.

3.2.6 What Barriers Exist to Being Able to Measure a Change in the Intermediate Outcomes Within the Next Two and a Half Years?

Conclusion: Potential barriers to measuring change in intermediate outcomes in the next two and a half years include inadequate time for changes to become apparent, insufficient resources and a lack of performance measurement information.

There was an expectation that intermediate outcomes (i.e., those that flow from the increased capacity already being demonstrated) could be measured in 2011, at the time of the five-year evaluation. These intermediate outcomes (Figure 1) are those expected to be felt by regulated industries.

Overall, the evaluation found limited feedback on barriers to measuring changes felt by industries in the next two and a half years. Among the interviewees who provided responses, the most common observations were that:

  • regulated industries may not perceive any changes by 2011; and
  • the current resource levels are inadequate to achieve the expected changes within this time frame.

Some of the intermediate outcomes can be partially measured by surveying the industries on their perceptions, such as whether the regulatory burden has decreased. However, the intermediate outcome of regulations increasingly meeting their intended objectives may be difficult to measure because of insufficient performance information.

In July 2009, the requirement to develop and implement Performance Measurement and Evaluation Plans for high-impact regulations was put in place. High-impact regulations produce 80% to 90% of the impacts resulting from regulations, but represent only about 10% of the regulations produced. Performance measurement and evaluation data for high-impact regulations alone would not provide adequate information about the overall outcomes achieved by regulations. The lack of a CDSR-specific performance measurement strategy is a barrier to a quality evaluation being conducted in 2011.

3.3 Efficiency and Economy

Efficiency generally means maximizing the outputs produced with a fixed level of inputs or minimizing the inputs used to produce a fixed level of outputs. Economy has a similar meaning, but with the intent to maximize expected outcomes, rather than outputs.

3.3.1 Is the Cabinet Directive on Streamlining Regulation Being Delivered Efficiently and Economically to Produce Desired Outputs and Outcomes?

Conclusion: The CDSR is demonstrating both efficiency and economy in producing the desired outputs and outcomes despite encountering funding restrictions and delays in funding. However, the degree of leveraging could not be determined, although it is implied given the sharing of tasks between TBS-RAS and departments and agencies.

Funds to begin the implementation of CDSR were expected in the spring of 2007–08, but were not received until October of that year. As a result, TBS-RAS had less time to begin the implementation of CDSR and lapsed $1 million in 2007–08. In 2008–09, TBS-RAS was subject to a spending freeze, resulting in a further lapse of $526,000. Table 3 provides an overview of spending; more detailed information is provided in Appendix F.

Table 3. Budgeted Spending, Actual Spending and Variance for TBS-RASa
  Budgeted Actual Variance
a.Includes A-base but excludes Employee Benefit Plan and accommodation charges.
2007–08 $3,749.4 $2,737.5 $1,011.9
2008–09 $4,978.6 $4,452.5 $526.1

Despite the delays in funding and limited resources, TBS-RAS managed to produce outputs such as reports, presentations, guides, tools and training (see par. 3.2.1) in the first two fiscal years. Many of the outputs have been produced at a lower cost than planned, thus demonstrating efficiency. Current gaps in the production of intended outputs appear to be caused by funding and resourcing issues, rather than inefficiency.

As noted in par. 3.2.1, performance reports are only available for the first two quarters of 2009–10. The limited performance data prevent the evaluation from fully assessing the economy of the CDSR. However, the data that do exist show that TBS-RAS has made progress in increasing the capacity of departments to meet CDSR requirements (i.e., the intended immediate outcome) at a lower cost than expected, thereby demonstrating economy.

Data were not available to quantify leveraging (i.e., the contribution of resources made by organizations relative to those provided through the CDSR). However, until summer 2009, the Centre of Regulatory Expertise and departments and agencies shared the cost of writing Regulatory Impact Analysis Statements, conducting cost-benefit analyses, and preparing Performance Measurement and Evaluation Plans, with a 30:70 split in effort. In addition, organizations are supplementing the work of TBS-RAS by creating tailored guides and using training budgets to send employees on CDSR-related training. This information indicates that leveraging was taking place, although its extent is not quantifiable.

3.3.2 Are There Any Alternative Design and Delivery Approaches That Should Be Considered for the Implementation of the Cabinet Directive on Streamlining Regulation? Is the Centralized Approach, With Some Cost-Sharing Funds for Departments and Agencies, an Efficient Model?

Conclusion: No alternatives to the current design and delivery approaches were identified. The experiences of countries belonging to the Organization for Economic Co-operation and Development (OECD) indicate that CDSR's centralized approach is an effective model for the implementation of regulatory policy.[6][7]

All of the interviewees from TBS-RAS and many of those from departments and agencies stated that the centralized approach used by TBS-RAS was the best one. Overall, departments and agencies value the advice and support provided by the Centre of Regulatory Expertise.

The experience of OECD countries shows that reforms to improve the quality of the regulatory system cannot be left solely to regulators, but that reforms can also fail if the system is too centralized. The OECD experience indicates that regulators must take primary responsibility, but within a system of incentives overseen by regulatory management and reform bodies. Such regulatory oversight bodies are often located in core government offices and have a mandate to ensure the quality of newly proposed rules and to develop programs for improving efficiency. Central oversight or coordination bodies encourage increased dialogue and interaction between the different ministries. Ultimately, however, the ministries themselves must commit to regulatory reform and ensure regulatory quality. In the Canadian context, TBS-RAS acts as the central oversight body for the CDSR, with the mandate to support ministers and departments in achieving regulatory efficiency and effectiveness. This role is consistent with the best practice centralized model identified through the international literature review.