ARCHIVED - Summative Evaluation of the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management

• Previous Page
• Table of Contents
• Next Page

This page has been archived.

5.0 Recommendations

1. Given that the Secretariat had various views on NLA, it would be beneficial for the department to develop a position on the NLA pilot project in light of the evaluation results and in relation to the other mechanisms. This process would facilitate a discussion and enable a decision regarding the proposed options. The discussion would address key questions including the following:
   • To what extent can the 5% carry-forward and reprofiling mechanisms be modified to build in the flexibility, timing, and predictability that resulted in the improved decision making when NLA was used? What are the benefits and risks of this?
   • Would enhancing existing mechanisms still result in improved decision making, financial management and project management?
   • What criteria should be used to qualify a department to participate in a new mechanism or enhancements to an existing mechanism?

   It is recommended that the Secretariat review existing mechanisms to determine whether they can be modified to build in the flexibility, timing and predictability that resulted in the improved decision making under the NLA pilot. Based on the results of the evaluation and the review, the Secretariat would then determine which of the options (2, 3 or 4) should be implemented.

   If the NLA pilot is continued or an existing alternative mechanism is enhanced (options 1 to 4), the following recommendations are made:

2. It is recommended that a performance measurement strategy be put in place with clear reporting requirements for participating departments.
   
   
3. Clear criteria for the selection of departments and design of projects for NLA should be developed and communicated. Criteria should be linked to the department's investment plan.
   
   
4. Roles and responsibilities between and within the Secretariat, the Department of Finance Canada and departments should be clearly defined, articulated and communicated.
   
   
5. SOPs regarding the functioning of the mechanism, rules and restrictions, and reporting requirements should be developed, articulated and communicated.
   
   
6. Departments should be reassessed for qualification whenever submitting a revised investment plan. Their capacity and track record (e.g., ability to carry out the projects in the investment plan, net and gross lapsing, MAF ratings, audit results, and results against indicators in the performance measurement strategy) should be carefully reviewed for this reassessment.

Appendix A—Interview Guides

Interview Guide for the Secretariat (including the Comptroller General) and the Department of Finance Canada

1. Please describe your familiarity, roles and responsibilities with respect to the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management.

   Relevance

2. Prior to the pilot, what would you identify as the most significant challenges for line departments in terms of managing capital assets?
   
   
3. From your perspective, has the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management addressed any of these challenges? If so, how?
   
   
4. Has the implementation of the pilot project posed any risks to the fiscal framework to date? Please explain.
   1. Do you anticipate any risks to the fiscal framework if the pilot were expanded to more or all departments?
   2. How could these risks be mitigated (e.g., by keeping it limited to: two-years, real property, or a certain amount within the SPA)?
   3. Are any of these risks unreasonable or unmanageable?

   Success

5. To your knowledge, to what extent did the pilot achieve its main objectives in regards to:
   1. Improved overall effectiveness of capital spending?
   2. Movement towards a long-term, strategic investment approach?
   3. The ability to respond to recommendations from the OAG regarding accrual concepts?
6. To what extent has the pilot allowed for improved financial management and expenditure practices? (probe for examples)
   
   
7. Has the pilot resulted in any outcomes (positive or negative) that were unforeseen?

   Implementation and design (Secretariat only)

8. To what extent were the Secretariat's roles and responsibilities well defined with regard to the implementation of the pilot? Did you feel the roles of participating departments were clear?
   1. How were these roles and responsibilities communicated (e.g., discussions with departments and the Secretariat, MOU, other documentation)?
   2. Were there any particular areas of confusion? (probe: were these areas of confusion tracked?) (specify)
   3. If so, how did the Secretariat respond to these challenges? (probe: how were these areas of confusion managed, how were the appropriate people made aware?)
9. To what extent were procedures and practices for the pilot well defined?
   1. How were these procedures and practices communicated? (e.g., discussions with departments and the Secretariat, MOU, other documentation).
   2. Were there any particular areas of confusion?(probe: were these areas of confusion tracked?) (specify)
   3. If so, how did the Secretariat respond to these challenges?(probe: how were these areas of confusion managed, how were the appropriate people made aware?)
10. Were the monitoring and reporting requirements appropriate? How could monitoring and reporting be improved?
    
    
11. Were there any (other) challenges with the implementation of the pilot project? How were these challenges addressed?
    
    
12. Was there anything about the design of the pilot that may have limited its ability to simulate non-lapsing two-year capital appropriations?

    Alternatives, best practices and lessons learned

13. What value does the pilot mechanism provide over and above existing mechanisms for managing capital budgets such as: (probe for specifics)
    1. The 5% carry forward?
    2. Reprofiling?
    3. Any other mechanisms for managing capital budgets?
14. If it were up to you, would you recommend that the pilot be continued, expanded, or dropped? Why?
    
    
15. If it were to be done over again, what would you suggest be done differently in implementing the pilot? What would you keep the same? (best practices and lessons learned).
    
    
16. If the pilot mechanism is continued or expanded to other departments, what would your message be to newly participating departments?
    
    
17. Does your experience with the pilot project hold any broader lessons in terms of the value of accrual concepts?
    
    
18. Do you have any other comments that you think may be useful to the evaluation?

Thank you for your participation.

Interview Guide for Participating Departments

1. Please describe your familiarity, roles and responsibilities with respect to the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management.
   
   
2. Please describe how the pilot was implemented at your department (i.e., size of SPA and coverage of asset categories).

   Relevance

3. Prior to the pilot, what would you identify as the most significant challenges for your department in terms of managing capital assets?
   
   
4. Has the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management addressed any of these challenges? If so, how?
   
   
5. Has the management of capital assets changed in your department since the introduction of the pilot? In what ways? What would your department do in its absence?

   Success

6. (Operational staff only.) Could you provide any examples from your department's experience with the pilot that demonstrate improved (probe: significance of the improvements, whether the improvements were a direct or indirect result of the pilot project):
   1. Project management practices?
   2. Risk management practices?
   3. Financial and expenditure management practices (e.g. budget planning, costing,  allocation and control, variance analysis and forecasting, and monitoring reporting)?
7. To what extent has the pilot allowed for a more strategic investment planning approach in terms of (probe for examples):
   1. Improved quality of decisions regarding capital asset management (e.g., lease versus buy)?
   2. Optimal allocation of resources?
   3. Enhanced value for money (i.e., were project and portfolio outcomes achieved in a cost effective manner)?
8. Has the pilot resulted in any outcomes (positive or negative) for your department that were unforeseen?

   Implementation and design

9. (Operational Staff only.) To what extent were the roles and responsibilities of participating departments well defined with regard to the implementation of the pilot? Did you feel that the Secretariat's roles were clear?
   1. How were these roles and responsibilities communicated (e.g., discussions with departments and the Secretariat, MOU, other documentation)?
   2. Were there any particular areas of confusion? (probe: were these areas of confusion tracked?) (specify)
   3. If so, how did your department respond to these challenges? (probe: how were these areas of confusion managed, how were the appropriate people made aware?)
10. (Operational Staff only.) To what extent were procedures and practices for the pilot well defined?
    1. How were these procedures and practices communicated? (e.g., discussions with departments and the Secretariat, MOU, other documentation)?
    2. Were there any particular areas of confusion?(probe: were these areas of confusion tracked?) (specify)
    3. If so, how did your department respond to these challenges? (probe: how were these areas of confusion managed, how were the appropriate people made aware?)
11. (Operational Staff only.) Were the monitoring and reporting requirements appropriate?
    
    
12. How could monitoring and reporting be improved? (probe: gaps, extra requirements, systems in place?)
    
    
13. Were there any (other) challenges with the implementation of the pilot project? How were these challenges addressed?
    
    
14. Was there anything about the design of the pilot which may have limited its ability to simulate non-lapsing two-year capital appropriations? (probe constraints on the amount of funds, timing, sources of funds)

    Alternatives, best practices and lessons learned

15. What value does the pilot mechanism provide over and above existing mechanisms for managing capital budgets such as: (probe for specifics)
    1. The 5% carry-forward?
    2. Reprofiling?
    3. Any other mechanisms for managing capital budgets?
16. (Operational Staff only.) If it were to be done over again, what would you suggest be done differently in implementing the pilot? What would you keep the same? (best practices and lessons learned).
    
    
17. If the pilot mechanism is continued or expanded to other departments, what would your message be to newly participating departments?
    
    
18. Does your experience with the pilot project provide any broad lessons about the value of accrual concepts?
    
    
19. Do you have any other comments that you think may be useful to the evaluation?

Thank you for your participation.

Interview Guide for Non-Participating Departments with a Non-Lapsing Mechanism

1. Please describe your familiarity, roles and responsibilities with respect to capital asset management at your department.
   
   
2. Please describe your familiarity with the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management.⁸ How did you become familiar with the pilot project?
   
   
3. Please describe your capital asset portfolio and planning processes. What amount does your department typically carry forward in terms of the percentage of capital budget? (probe for level of delegated authority, the number and scope of capital projects per year, the status of the Long-Term Capital Plan or Investment Plan).
   
   
4. Please describe the extent to which current mechanisms (100% carry-forward and reprofiling) may allow you to appropriately manage your capital asset portfolio? Do these mechanisms meet your needs?
   
   
5. Overall, what would you identify as the most significant challenges for your department in terms of managing capital assets?
   
   
6. Does the ability to carry-forward 100% of your capital budget address any of these challenges? If so, how?
   
   
7. How else does the carry-forward mechanism affect capital asset management at your department in terms of:
   1. Improved quality of decisions regarding capital asset management (e.g., lease versus buy)?
   2. Optimal allocation of resources?
   3. Enhanced value for money (project and/or portfolio outcomes achieved in a cost effective manner)?
   4. Other ways to allow for a more strategic investment planning approach?
   5. Other (reporting burden, process issues, uncertainty)?
8. With regards to capital asset management, what are some best practices your department has developed or adopted with respect to:
   1. Project management practices?
   2. Risk management practices?
   3. Financial expenditure management practices (e.g., budget planning, costing, allocation and control, variance analysis and forecasting, and monitoring and reporting)?
9. Do you have any other comments that you think may be useful to the evaluation?

Thank you for your participation.

Interview Guide for Non-Participating Departments without a Non-Lapsing Mechanism

1. Please describe your familiarity, roles and responsibilities with respect to capital asset management in your department.
   
   
2. Please describe your familiarity with the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management.⁹ How did you become familiar with the pilot project?
   
   
3. Please describe your capital asset portfolio and planning processes. What amount does your department typically carry forward in terms of the percentage of capital budget? (probe for level of delegated authority, the number and scope of capital projects per year, the status of the Long-Term Capital Plan or Investment Plan).
   
   
4. Please describe the extent to which current mechanisms (5% carry-forward and reprofiling) may allow you to appropriately manage your capital asset portfolio? Do these mechanisms meet your needs?
   
   
5. Overall, what would you identify as the most significant challenges for your department in terms of managing capital assets?
   
   
6. Would the introduction of a two-year non-lapsing mechanism address any of these challenges? If so, how?
   
   
7. How else might a two-year non-lapsing mechanism affect capital asset management at your department in terms of:
   1. Improved quality of decisions regarding capital asset management (e.g., lease versus buy)?
   2. Optimal allocation of resources?
   3. Enhanced value for money (were project and portfolio outcomes achieved in a cost effective manner)?
   4. Other ways to allow for a more strategic investment planning approach?
8. With regards to capital asset management, what are some best practices your department has developed or adopted with respect to:
   1. Project management practices?
   2. Risk management practices?
   3. Financial expenditure management practices (e.g., budget planning, costing, allocation and control, variance analysis and forecasting, and monitoring and reporting)?
9. If the pilot project were expanded to include your department, how would this affect the way in which you currently manage your capital assets?
   
   
10. Do you have any other comments that you think may be useful to the evaluation?

Thank you for your participation.

• Previous Page
• Table of Contents
• Next Page