ARCHIVED - Summative Evaluation of the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management

• Previous Page
• Table of Contents
• Next Page

This page has been archived.

1.0 Introduction

This report represents the results of the Evaluation of the Pilot Project on Non-Lapsing Appropriations (NLA) for Capital Asset Management that was undertaken in 2009–10.

The NLA pilot project was launched in 2005 as part of the response to recommendations made by two reports. The first was a 2004 Capital Asset Review (CAR) conducted under the umbrella of Government Expenditure Reviews in December 2003. The second was a 2004 report from the OAG that strongly recommended introducing accrual appropriations for federal expenditures. The purpose of the pilot was to study the potential impact of a possible government-wide application of a two-year NLA carry-forward mechanism.

The evaluation was overseen by the Secretariat's project authority at the Real Property and Materiel Policy Division (RPMPD), the Working Group for the Pilot Project on NLA for Capital Asset Management, and the Secretariat's Internal Audit and Evaluation Bureau. The evaluation was required to be completed as per the 2006 Treasury Board submission.

1.1 Background/Context

1.1.1 Overview

By providing the required structures, land, machinery and equipment, capital assets are a fundamental support to the majority of government programs. Over the years, issues concerning capital asset management in the federal government have emerged. Insufficient investment in capital assets has led to deteriorating infrastructure (i.e., "rust out"). A review was undertaken to examine expenditures and business processes and to identify opportunities for improvement in management, governance and effectiveness. The final CAR report, published in 2004, stated the following:

Though the management of assets is generally well done at the local level, the most costly item in this domain is deferred maintenance. It has been estimated by reliable sources that deferral of maintenance can cost up to five times the cost of doing maintenance on time…. The savings are found in both the repair and operating costs.

Following logically from this finding, the report went on to recommend that the federal government "provide non-lapsing asset funding over a 3 to 5 year time frame, contingent upon the quality of the long-term asset plan and department performance." The concept of "lapsing" refers to unspent budget at the end of the fiscal year-end. Under current processes at fiscal year-end, unspent funds in a department's budget are no longer available to the department. Instead, these funds are redirected into a government-wide consolidated account and spent on government priorities (e.g., paying down the national debt) or reallocated to other program areas. A "non-lapsing appropriation," then, would be a portion of a department or agency's budget that, if unspent at year-end, can be carried forward into the following fiscal year.

Also in 2004, a report from the OAG strongly recommended introducing accrual appropriations for federal expenditures. In this context, NLA for capital assets may be viewed as part of a possible broader vision to optimize federal government asset management.

1.1.2 Description of the pilot project

The NLA pilot was launched in 2005 as a step toward implementing these recommendations. The purpose of the pilot was to study the potential impact and effects of a government-wide application of a carry-forward mechanism. The pilot project had three main objectives:

• Improved overall effectiveness of capital spending (including improved management decisions and enhanced value for money);
• A move toward a longer-term, strategic investment approach consistent with the nature of the asset; and
• An ability to respond to recommendations from the OAG regarding accrual accounting (i.e., the OAG recommended that departments improve and use accrual financial information for daily decision making).

The pilot project was designed to simulate non-lapsing two-year capital appropriations. That is, it would allow amounts of unspent capital budgets at year-end to be added to the department's budget for the next fiscal year. Participating departments included Agriculture and Agri-food Canada (AAFC), Foreign Affairs and International Trade Canada (DFAIT), Royal Canadian Mounted Police (RCMP), and Fisheries and Oceans Canada (DFO).² These departments were given the opportunity to carry forward 100% of their unexpended capital vote appropriations from the current fiscal year to the one following.

With the exception of DFAIT, all amounts carried forward were to be solely used for capital expenditures. The amounts carried forward by DFAIT were even further restricted to real property expenditures, although this restriction was lifted for fiscal year 2008–09 to include all capital. In this pilot, DFAIT was also limited to carrying forward a maximum of 5% of their capital vote (real property funds only in the first three years of the pilot and all capital funds, except Mission Security, since 2008–09).

Participating departments were chosen based on the following criteria:

• The department or agency must have completed a Long-Term Capital Plan that was approved by the Secretariat;
• The department or agency must have a record of sound financial management; and
• The Secretariat must support the organization's participation in the pilot.

Memoranda of Understanding (MOUs) signed between the Secretariat and the participating departments identified the amounts eligible for carry-forward in the pilot (see Exhibit 1.1).

Exhibit 1.1—Eligible Amounts to be Carried Forward

Departments	AAFC	DFAIT	DFO	RCMP
Amount for Carry-Forward	$30 million	$78 million	$141.8 million	$20 million
Notes		Capped at 5% of the department's capital vote. Carry-forward limited to real property assets. As of 2008–09, all capital, with the exception of Mission Security, was eligible for carry-forward under this pilot.	DFO did not join the pilot until November 2007.	RCMP received one-time permission from Treasury Board to add $53 million to their special purpose allotment (SPA) in fiscal year 2007–08.

The final year from which appropriations could be carried forward was fiscal year 2009–10. Therefore, the final year for which funds could be included in Supplementary Estimates (A) is fiscal year 2010–11.

Special purpose allotments

SPAs were established for fiscal years 2005–06 to 2009–10 for each participating department for the amounts specified in Exhibit 1.1. Authority was granted to these departments to carry forward the unexpended balance of the SPAs established for Capital Asset Management in the previous fiscal year and to increase the SPAs by the amount carried forward. This was done through the inclusion of items in Supplementary Estimates.

Reporting requirements

The MOUs identified the reporting requirements of participating departments in two broad categories: financial results and impact on management decisions.

Specifically, departments were expected to provide details of financial carry-forward on a project-by-project basis. As well, departments were required to provide descriptive information regarding the impact on their management decisions by illustrating how the ability to carry forward affected their investment and/or business planning (including up to five specific examples).

Further guidance was provided in June 2009 to departments in the form of a template for the annual report. This template required that departments report on both a cash and an accrual basis. It also required the identification of specific projects for which carry-forward amounts were associated. A narrative section asked participating departments to explain how the pilot project had affected the management of capital assets more generally and also asked for descriptive examples of how specific projects were impacted by the pilot.

Roles and responsibilities

MOUs with the participating departments outlined the following roles and responsibilities for the Secretariat and the departments in the implementation of the pilot project.

The Secretariat had responsibilities for:

• Seeking appropriate Treasury Board authority on behalf of the participating departments for non-lapsing capital appropriations;
• Modifying SPAs to operationalize these appropriations, as required;
• Working with the participating departments to identify benefits and solutions to address barriers which may arise during the pilot;
• Tracking capital funds eligible for carry-forward through figures provided to the Public Accounts Committee;
• Ensuring that any funds transferred into the capital vote during the fiscal year were not eligible for carry-forward; and
• Releasing non-lapsing funds to departments through Supplementary Estimates (A).

The participating departments had responsibilities for:

• Seeking ministerial support to participate in the pilot;
• Determining the funds eligible for carry-forward;
• Ensuring that funds eligible for carry-forward were fenced for the duration of the pilot and redirected to capital spending;
• Ensuring that reprofiled funds were displayed by project on an accrual and a cash basis; and
• Meeting agreed-upon reporting requirements.

RPMPD was the lead for this pilot project at the Secretariat. Responsibility to support the management of the pilot was shared by the Secretariat's Program Sector and Expenditure Management Sector.

1.1.3 Logic model for the pilot project

A logic model was developed near the beginning of the pilot to describe how the NLA activities were expected to lead to the outputs and the desired immediate, intermediate and ultimate outcomes. The logic model was used as the basis for developing evaluation questions to address the performance of the program.

Exhibit 1.2—Logic Model for the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management

Exhibit 1.2 - Text version

1.1.4 Other mechanisms for transferring capital funds

In order to fully understand the context for the pilot project on NLA, it is useful to review the other mechanisms that were already available to departments and agencies in managing their capital votes. What follows is a discussion of two of these mechanisms along with their key features.

5% capital carry-forward

The 5% capital carry-forward allows eligible departments and agencies to carry forward capital budget amounts from one fiscal year to the next, up to 5% of their capital vote to a maximum of $75 million. The origin of the 5% carry-forward was Treasury Board Circular 1987-53, whose intent was to "give departments flexibility in managing capital funds by allowing them to carry forward limited amounts of these funds into the next fiscal year." This mechanism applies to departments and agencies that have separate capital votes in the Main Estimates, but it does not apply to Crown corporations or to National Defence.

The information requiredto request a carry-forward includes proposals that must be related to specific capital plans, objectives and results and must be justified in relation to the principles outlined in the Circular. Departments must clearly explain the circumstances around the funds that are expected to lapse, demonstrate the need to carry moneys forward, and explain how these would eventually be spent.

By December 15 of each year, deputy heads must write to the Secretary of the Treasury Board to seek frozen allotments equal in amount to the desired carry- forward. Approved amounts are frozen in the current fiscal year, with departments authorized to include them in the Supplementary Estimates for the following year.

Capital reprofiling

The capital reprofiling mechanism allows departments, agencies, and Crown corporations supported by appropriations to carry forward capital budget amounts from one fiscal year to a future fiscal year. In late August, a Secretariat call letter solicits Annual Reference Level Update (ARLU) and reprofile requests from organizations. Although there is no explicit limit on reprofile amounts, requests must meet eligibility requirements and are subject to certain conditions. Requests are then assessed by the Secretariatand the Department of Finance Canada. Decisions are usually made in December, and the approved amounts are included in a future year Supplementary Estimates and frozen in the current year.

1.2 Objectives

This evaluation sought to achieve four main objectives (from which flow the four main evaluation issues):

• Assess the relevance of two-year non-lapsing appropriations;
• Assess the appropriateness of the overall implementation and design of the pilot project;
• Determine the extent to which the pilot project has achieved immediate outcomes and has made progress toward intermediate outcomes (i.e., success); and
• Document best practices and lessons learned and identify possible alternatives.

Because the pilot ended in 2009–10 (i.e., this was the last fiscal year from which appropriations could be carried forward), the evaluation provides options to address the key question of what should follow the pilot project. It is expected that the evaluation will be used as one source of information in developing a future mechanism that facilitates the management of federal government capital programs.

1.3 Evaluation Issues and Questions

The evaluation provides findings to questions under four main evaluation issues: relevance, implementation and design, success³ and alternatives, and best practices and lessons learned.⁴

Relevance

• 1. Is the use of non-lapsing appropriations consistent with federal government priorities for capital asset management?
  
  
• 2. Is there a continuing need for non-lapsing appropriations for capital asset management?

Success

• 3. To what extent has the pilot project led to improved financial management of capital asset budgets?
  
  
• 4. To what extent has the pilot project led to improved management of capital asset projects?
  
  
• 5. To what extent has the pilot led to improved risk management in allocating resources from the capital budget?
  
  
• 6. To what extent has there been progress toward achieving intermediate outcomes?
  1. Improved quality of decisions
  2. Enhanced value for money
  3. Optimal allocation of resources in the investment plan
• 7. How does the achievement of these outcomes vary by participating department? To what extent can variations be explained by differences in how the pilot project was implemented at each department (i.e., proportion of capital vote in the SPA, length of participation in the pilot)?
  
  
• 8. Given what is known about progress toward immediate and intermediate outcomes, how likely is it that this mechanism can achieve ultimate outcomes?
  
  
• 9. Were there any unintended outcomes of the pilot project (negative or positive)?

Implementation and design

• 10. To what extent were roles and responsibilities for the pilot project clearly defined and articulated?
  
  
• 11. To what extent were the standard operating procedures and practices for the pilot project well defined, documented and understood? What were the particular areas of concern or confusion?
  
  
• 12. What were the main challenges encountered during the implementation and management of the pilot project?⁵
  
  
• 13. How could monitoring and reporting of the pilot project be improved? To what extent are data currently available to assess the overall success of the pilot project? To what extent did participating departments provide accrual-based reports?

Best practices, lessons learned and risks

• 14. What best practices were observed by participating departments?
  
  
• 15. What were the main lessons learned from the pilot project? How can these be applied to future pilot projects generally and to the possible continuation and expansion of the pilot project for non-lapsing appropriations for capital asset management?
  
  
• 16. Does the concept of non-lapsing appropriations pose any unreasonable or unmanageable risks for the fiscal framework?

Alternatives

• 17. Is there an alternative mechanism (including existing mechanisms) that would achieve similar outcomes for departments in managing their capital budgets?

1.4 Methodology

Three main lines of evidence were employed for this evaluation: document review, key informant interviews, and administrative data review.

1.4.1 Document review

A document review was conducted using a template developed in the design phase of the evaluation. In all, 45 documents were reviewed, public documents (n=13), decision documents (n=19), and monitoring and reporting documents (n=13).

1.4.2 Key informant interviews

In all, 20 interviews with 33 individuals were conducted for the evaluation as follows:

• Seven interviews with 10 individuals representing participating departments;
• Nine interviews with 15 individuals representing the Secretariat and the Office of the Comptroller General of Canada;
• One interview with two individuals representing the Department of Finance Canada;
• Two interviews with three individuals representing non-participating departments with 100% carry-forward; and
• One interview with three individuals representing non-participating departments with 100% carry-forward.

Interviews were semi-structured and followed an interview guide tailored to each respondent group (see Appendix A).

1.4.3 Administrative data review

The administrative data review included all financial information available to populate the template on expenditure management data (presented by department in Appendix B) and was done in two parts. The participating departments' financial information for fiscal years 2005–06 to 2009–10 was gathered from the Public Accounts of Canada, which are posted on the Library and Archives Canada website. Subsequently, the financial data contained in the template were verified and completed by departments. The intention was that once the template was fully populated, the financial information (i.e., amount of carry-forward, amount reprofiled, amount of capital funds lapsed) would be analyzed with a view to responding to the evaluation questions on the success of the pilot project.

The second component of the administrative data review involved gathering financial information on capital project spending. This information was obtained from the Departmental Performance Reports (DPRs) for fiscal years 2003–04 to 2008–09 for all participating departments. These data were gathered for several years prior to pilot implementation in order to identify trends from pre- to post-pilot implementation. Projects identified in the NLA Annual Reports were cross-referenced with the information contained in the DPRs in order to focus on projects that utilized the NLA mechanism. The intention of this analysis was to examine the extent to which the pilot project influenced decision making around capital asset management. Further, the analysis assessed whether the pilot project allowed participating departments to make different decisions in carrying out capital projects than they would have made before participating in the pilot.

1.5 Limitations

Several limitations of the study affect the interpretation of the findings:

• Inconsistency in how the NLA pilot project was applied to participating departments restricted the ability of the evaluation to make direct comparisons of the pilot's impact across departments, though qualitative evidence was used to identify key trends and themes.
• Turnover in key positions at the Secretariat has meant that many of the individuals responsible for the launch of the pilot were unavailable. Data regarding challenges faced during the earliest stages of implementation have largely been gathered from indirect sources.
• Some documentation from the earlier years of the pilot was simply unavailable or non-existent, such as annual reports from participating departments for the 2005–06 fiscal year. Data were taken instead from subsequent annual reports and DPRs. In addition, as templates for reporting were not disseminated to departments until 2009, annual reports prior to this are of lower quality and detail.
• Financial information received from departments often did not align with the same information found in the Public Accounts of Canada. These discrepancies were noted, and financial information used was verified through multiple rounds of cross comparison.
• Information on accrual reporting was not available since departments did not report on an accrual basis.

There was an expectation in the evaluation design that the administrative data would contribute to the findings for many evaluation questions relating to success. However, after a careful review of the various sources, the evaluators found it difficult to draw meaningful conclusions from the financial data. The conclusions of the evaluation are based largely on the qualitative evidence.

1.6 Organization of the Report

Findings and conclusions are presented in Section 2.0. To facilitate the flow of the report, evaluation questions have been reordered. Section 3.0 presents the options for moving forward. Section 4.0 presents a summary of conclusions while Section 5.0 presents the recommendations.

• Previous Page
• Table of Contents
• Next Page