Regional Development Agencies Audit of the Management Control Framework over Monitoring and Reporting for Selected Grants and Contributions Programs

Table of Contents

Executive Summary

The objective of this audit was to provide reasonable assurance that selected Regional Development Agencies (RDAs) have a sound management control frameworkFootnote 1 to manage the monitoring and reporting of grants and contributions in compliance with the Treasury Board Policy on Transfer Payments and Directive on Transfer Payments.

Why This Is Important

Grants and contributions programs are designed to provide assistance to local governments, non-profit organizations and businesses in order to advance technologies, develop products and stimulate local economies.

The successful delivery of grants and contributions programs is a key priority for RDAs, whose main objectives are to have successful, diversified and participative local economies for the benefit of all Canadians.

Monitoring and reporting are essential program management control elements within the transfer payment process. RDAs need an effective management control framework over monitoring and reporting activities, including payments to recipients to ensure compliance with program and funding agreement terms and conditions. Such frameworks help to support compliance with the Treasury Board Policy on Transfer Payments and related directive, and legislative requirements, including the Financial Administration Act (FAA).

Key Findings

Although RDAs share a common mandate of supporting economic development within their respective regions, their approaches to managing and delivering grants and contributions programs vary.

The scope of this audit examined the Atlantic Canada Opportunities Agency (ACOA) – Business Development Program (BDP), the Economic Development Agency of Canada for the Regions of Quebec (CEDQ) – Quebec Economic Development Program (QEDP), and the Federal Economic Development Agency for Southern Ontario (FedDev Ontario) – Southern Ontario Development Program (SODP): Southern Ontario Advantage Initiatives (SOAI).

For the selected RDA programs, the scope of the audit examined: program governance; the development and implementation of monitoring and reporting processes; payments to and reimbursements from recipients; and relationship management activities with program stakeholders to improve their programs.

Overall, ACOA, CEDQ and FedDev Ontario have established management control frameworks to manage the monitoring and reporting processes for their contribution programs. The three RDAs have established program governance structures and internal controls in consideration of risks to support compliance and Agency-wide monitoring and reporting requirements. All three RDAs are examining and using program cost information to support program and other organizational decision making with regard to oversight, monitoring, reporting, and payment activities, and in support of budgeting and resource estimates. All three RDAs have also defined and communicated program management and delivery requirements, and are managing their identified monitoring and reporting risks.

All three RDAs have developed monitoring and reporting requirements that consider program risk and results, and comply with the Policy on Transfer Payments, Directive on Transfer Payments and applicable legislation. The three RDAs have established performance measurement strategies aligned with their Program Alignment Architectures (PAAs), Reports on Plans and Priorities (RPPs) and Departmental Performance Reports (DPRs), along with approaches to assign monitoring and reporting requirements to recipients based on their risk assessment ratings. ACOA and FedDev Ontario can improve the consistency of documenting the evidence of review of recipient progress reporting to ensure compliance with program and funding agreement terms and conditions.

All three RDAs are monitoring payments to, and reimbursements from recipients to ensure they comply with program and funding agreement terms and conditions, the Policy on Transfer Payments and the FAA. All three RDAs have established monitoring procedures and controls for payment verification and approval, recipient repayment compliance, and the closure of projects. ACOA and CEDQ can improve the consistent performance and documentation of their payment verification controls.

All three RDAs have engaged with their stakeholders to receive feedback to improve program performance. FedDev Ontario is addressing opportunities to improve its management of stakeholder relationships and program performance.

Conclusion

Overall, the management control frameworks over monitoring and reporting established and implemented by ACOA, CEDQ and FedDev Ontario support compliance with the Policy on Transfer Payments and the Directive on Transfer Payments.

Conformance with Professional Standards

This audit engagement was conducted in accordance with the Internal Auditing Standards for the Government of Canada based on the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing. A practice inspection is currently underway with results expected before the end of 2013-14

Anthea English, CPA, CA
Assistant Comptroller General and Chief Audit Executive
Internal Audit Sector, Office of the Comptroller General

Audit Objective and Scope

The objective of this audit was to provide reasonable assurance that selected RDAs have a sound management control framework to manage the monitoring and reporting of grants and contributions in compliance with the Treasury Board Policy on Transfer Payments and the Directive on Transfer Payments.

The Audit of Monitoring and Reporting of Selected Grants and Contributions Programs was identified and approved in the RDA Risk-Based Internal Audit Plan (RBAP) for 2012–13. Through consultations with the RDAs on the 2012-13 RBAP, the following three RDA transfer payment programs were selected for the audit based upon their size and materiality: the ACOABDP, the CEDQQEDP, and the FedDev OntarioSODP: SOAI. Appendix C provides additional information on the selected RDA programs.

The scope of the audit examined: RDA program governance; the development and implementation of monitoring and reporting processes; payments to and reimbursements from recipients; and relationship management activities with program stakeholders to improve their programs. The audit tested RDA verification and approval controls related to payment and reimbursement transactions to ensure that program terms and conditions, the Policy on Transfer Payments and FAA requirements were met.

For ACOA and FedDev Ontario, the audit sample selected repayable and non-repayable funded BDP and SODP: SOAI projects between fiscal years 2010–12. For CEDQ, the audit sample selected repayable and non-repayable funded projects between fiscal years 2006–07 to 2012–13. The CEDQ audit sample also examined selected infrastructure program projects that are managed directly by the Agency as well as infrastructure programs managed in partnership with Infrastructure Canada and the province of Quebec which are governed by Memoranda of Understanding (MOUs).  The audit did not examine the performance of these MOUs.

The lines of enquiry and audit criteria are provided in Appendix A.

Detailed Findings and Recommendations

Finding 1: Program Governance

Program governance is a process through which organizations set strategic directions, define and assign responsibilities, assess risks, and implement measures to achieve outcomes consistent with established objectives. When expectations are clearly defined and communicated, the likelihood increases that the management and delivery of program objectives will be successful and achieved in consideration of accountability for performance, results and risks.

The audit examined the extent to which the RDAs have established program governance, risk management, and internal control processes to monitor recipient compliance and to support RDA-wide monitoring and reporting requirements.

All three RDAs have established risk-based oversight mechanisms and internal controls to support program monitoring and reporting. Program cost information is being used by the three RDAs to support decision making.

The audit found the oversight responsibilities of the RDA committee structures examined included the coherent implementation of program governance, risk management, financial and program performance measurement, monitoring and reporting, and client engagement activities. Each RDA was found to have established and implemented contribution program management processes, procedures and internal controls related to monitoring and reporting that were found to be risk-based. This approach complies with the requirements of the Policy on Transfer Payments and Directive on Transfer Payments.

The audit also found that the RDAs are examining the costs of contribution program delivery and are using this information to support program delivery and other organizational decision making. ACOA and CEDQ are analyzing the costs of delivering their contribution programs with regard to oversight, monitoring, reporting and payment activities, and in support of budgeting and resource estimates. FedDev Ontario has initiated an activity-based costing review for its contribution program delivery.

All three RDAs have defined and communicated policies, procedures, controls and clear roles and responsibilities over program monitoring and reporting processes.

Program representatives from each RDA indicated that the roles, responsibilities and performance expectations over monitoring, reporting and other contribution program management activities have been clearly defined and communicated through policies, procedures and training. For example, the RDAs have provided training sessions related to due diligence, risk management, financial statement analysis and managing federal contribution programs. Each RDA program employee develops annual performance and learning plans that are reviewed semi-annually to support the performance of their roles and responsibilities.

All three RDAs have identified organizational risks related to monitoring and reporting, and have developed strategies to manage these risks.

Through their risk management activities, the three RDAs have identified risks related to monitoring and reporting, and have established strategies to monitor these risks.

ACOA uses an Executive Dashboard report to provide timely decision-making information related to trends in performance against targets to provide and support program-level decision-making information, risk management and results reporting. CEDQ has established risk-based recipient monitoring, reporting, and payment verifications processes, along with performance measurement strategy to analyze the achievement of project results and to report on project/program performance. FedDev Ontario management is currently examining strategic organizational and program level priorities and opportunities, including the Agency’s risk management governance structure, information systems and data quality initiatives, to improve information for program performance and results reporting.

Overall, the three RDAs have established program governance, risk management and internal controls to monitor recipient compliance and to support Agency-wide monitoring and reporting requirements.

Finding 2: Monitoring and Reporting Processes

The Policy on Transfer Payments requires that transfer payment programs be managed with integrity, transparency and accountability in a manner that is sensitive to risks; citizen- and recipient-focused; and designed and delivered to address government priorities in achieving results for Canadians.

The audit examined the extent to which the RDAs have implemented program monitoring and reporting requirements in consideration of program risks, results, and compliance with the Policy on Transfer Payments, its related directive and applicable RDA legislation.

All three RDAs have designed performance measurement strategies to support program results reporting.

For the programs selected, each RDA has designed performance measurement strategies with measures, indicators and data collection strategies to support program results reporting. The performance measurement strategies examined were found to be in alignment with the RDA’s PAAs. Each RDA has program performance information systems in place to support PAA-based results reporting through their RPPs and DPRs.

All three RDAs have established approaches to assign recipient monitoring and reporting requirements based on risk.

Each RDA has established methodologies to assign monitoring and reporting requirements on a basis that is proportionate to the recipient/project risk assessment rating. For the recipient funding agreements reviewed, reporting requirements included project progress reporting, financial statements, or both.

The review of sampled ACOA files found that some funding agreements did not contain the required type and/or frequency of reporting requirements in relation to recipient risk assessment rating. In response, management indicated that the Agency is in the process of implementing new funding agreement templates to address this finding. The review of sampled CEDQ files (with one exception) and FedDev Ontario files found that all recipient contribution agreements contained the appropriate type and frequency of monitoring and reporting requirements in relation to their project risk assessment rating.

The RDAs followed-up with recipients in instances of non-compliance with funding agreement reporting requirements.

The review of sampled RDA files found that for each RDA some recipients did not always provide their reporting in accordance with the content, format, and/or timeline requirements specified in their funding agreements. However, evidence was found demonstrating that RDAs followed-up with recipients to obtain the required reporting information.

The extent to which the RDAs documented evidence of their review of recipient reporting requirements varied between the RDAs.

The review of sampled files found that the RDAs are, for the most part, documenting evidence of their review of recipient financial statement reporting and progress reporting submissions in accordance with their established program guidance.

For sampled ACOA files where recipients were required to provide financial statements, the required monitoring report to document the review and analysis of recipient financial information was used. This report is also used to assess the overall state of the project file and to determine whether a change in the recipient’s risk rating is needed. The review of sampled ACOA files also found some exceptions where evidence of review of recipient progress reporting was not always clearly documented. It was also noted that different methods are used by ACOA to review and assess recipient compliance with progress reporting requirements.

The review of all sampled CEDQ files found evidence of review of recipient financial statements, progress reporting or other reporting deliverables.

The review of sampled FedDev Ontario files found some exceptions where evidence of review of recipient progress reporting was not always clearly documented. It was also noted that different methods are used by FedDev Ontario to review and assess recipient compliance with reporting requirements.

Documenting the review of recipient reporting deliverables and other program terms and conditions demonstrates the program’s due diligence in assessing that recipients are achieving intended results and helps detect potential issues with the recipient and/or project. The consistent review of recipient reporting deliverables helps to detect recipient financial / non-financial risks and non-compliance with funding agreements. There are also potential efficiencies to be gained by using a standard approach, rather than different methods, to reviewing recipient reporting to ensure recipient compliance with funding agreement requirements and program terms and conditions.

Overall, all three RDAs have developed and managed monitoring and reporting requirements in consideration of program risk, results, and in a manner to support compliance with the transfer payments policy, directive and RDA legislation.

ACOA and FedDev Ontario can improve the documentation of the review of recipient progress reporting requirements to ensure compliance with funding agreement terms and conditions.

Recommendation 1

ACOA and FedDev Ontario management should take steps to ensure evidence of review of recipient progress reporting requirements is consistently documented.

Finding 3: Payments and Reimbursements

The Policy on Transfer Payments requires payments to be made to recipients in a timely, prudent and efficient manner that supports the achievement of objectives, in consideration of risks, and ensure that recipients are in compliance with their funding agreements. The policy also requires that amounts repayable by or recoverable from recipients be recognized as debts due to the Crown and that appropriate action be taken for recovery.

The audit examined the extent to which payments made by the RDAs to recipients, along with recipient reimbursements to the RDAs, are monitored to ensure they have been performed in compliance with funding agreement terms and conditions, the Policy on Transfer Payments, its related directive, and the FAA.

All three RDAs have established monitoring processes and controls for the review and approval of recipient payments. Exceptions were observed with regard to the consistency in performing specific payment review and verification controls.

The audit found the RDAs use of a number of appropriate payment practices including: risk-based processes, controls and forms to verify and approve recipient claims; delegation of financial signing authorities’ matrices establishing contribution program authorities under the FAA; and centralized finance functions for payment processing. The review of payments made under the three programs found that all sampled payments were verified and approved by individuals with the appropriate delegated authorities under the FAA Section 34 and Section 33, and that there was a separation of the duties between the individuals approving the payment (Section 34) and processing the payment (Section 33).

ACOA and CEDQ use a risk-based post-payment audit process and are monitoring payment processing service standards. FedDev Ontario is in the process of developing such service standards. These service standards are important measures to assess the effectiveness of payment review and verification processes and controls. FedDev Ontario uses a sampling summary form that is signed by a manager as a quality assurance control to ensure that the required level of risk-based verification of invoices for costs claimed and proof of payment was performed.

The review of ACOA payments found some exceptions where there was not clear documentation of payment review and verification controls and one exception where an ineligible expense was reimbursed. This exception was not detected by ACOA’s payment review and verification controls. The review of CEDQ payments found a few exceptions where recipients were reimbursed for costs prior to the funding agreement eligibility date and one exception where an ineligible expense was reimbursed. These exceptions were not detected by CEDQ’s payment review and verification controls. ACOA and CEDQ should ensure that payment review and verification controls and performed and documented consistently to ensure that recipients are reimbursed only for the expenses they are entitled to per their funding agreements.

The review of sampled FedDev Ontario payments found no exceptions with the performance of the Agency’s payment review and verification controls.

All three RDAs have established procedures for managing and monitoring recipient compliance with repayment terms and conditions.

The audit found that each RDA has established guidance, procedures and controls to manage recipient risks of not reimbursing their contributions through funding agreement repayment provisions. RDA guidance and procedures were found to be in alignment with the Treasury Board Directive on Receivables Management.

Each RDA is monitoring and reporting on recipient compliance with funding agreement repayment terms and conditions. All funding agreements identified consequences in the event of recipient default and/or non-compliance with funding agreement terms and conditions. All sampled repayable funding agreements contained a repayment schedule.

ACOA compares and benchmarks its default and write-off percentages with industry trends reported by the Business Development Bank of Canada (BDC) and CEDQ. Regular communication occurs between the regional offices and headquarters staff to discuss recipient files, and annual Agency-wide meetings are held to share lessons-learned. The review of sampled ACOA files found that where applicable, most recipients had met the repayment requirements established by their funding agreements, and in those cases where they had not, ACOA was pursuing the recovery of these funds in accordance with its collections and recovery process.

CEDQ compares and benchmarks its default and write-off percentages to data reported by the Office of the Superintendent of Financial Institutions (OSFI). CEDQ also prepares a balanced scorecard report to help mitigate recipient default and insolvency rates within its contribution program portfolio. All sampled CEDQ recipients met their repayment requirements. 

Testing the performance of FedDev Ontario’s receivables management process was not possible at the time of the audit as no SOAI projects had reached the point where the recipients were required to begin repayment of their contributions. No SOAI projects were in default at the time of the audit.

The RDAs have established project closure procedures and controls. All sampled projects were closed in compliance with established RDA procedures and controls.

Each RDA has established closure procedures and controls to manage recipient project files that have been completed and/or have been found in default with funding agreement terms and conditions. All the RDAs have adopted processes and measures to help protect federal funding in the event of recipient default by pursuing available options to recover funding prior to project closure and/or a write-off decision.

The review of sampled ACOA and CEDQ default projects found that each had applied required recovery procedures to the extent possible, and documented the chronology of events in accordance with their established procedural requirements. For all sampled CEDQ default files, the audit procedures confirmed that the recipients had repaid their contributions prior to project closure. Sampled CEDQ write-off decisions were documented appropriately and approved by delegated authorities. Testing the performance of FedDev Ontario’s project closure and receivables management process was not possible at the time of the audit as no SOAI projects were completed and/or found to be in default.

Overall, the RDAs are monitoring payments to and reimbursements from recipients to ensure they comply with terms and conditions, the Policy on Transfer Payments, and the FAA. All three RDAs have established monitoring procedures and controls for payment verification and approval, recipient repayment compliance, and the closure of projects.

Opportunities for improvement were identified for ACOA and CEDQ with regard to the consistency of documenting the performance of and/or performing specific payment review and verification controls.

Recommendation 2

ACOA and CEDQ management should ensure that required payment verification and approval activities are performed and documented consistently with expected procedures and controls. ACOA and CEDQ should also take appropriate actions to recover the ineligible amounts paid to recipients.

Finding 4: Managing Relationships with Stakeholders and Continuous Improvement

The Policy on Transfer Payments requires federal organizations responsible for transfer payment programs to manage these programs in consideration of their recipients. The audit examined the extent to which the RDAs have established client relationship management mechanisms to engage with its recipients to achieve intended results and to measure performance in support of continuous program improvement.

Most RDAs have established formal relationship management practices to receive stakeholder feedback to improve program performance.

Each RDA has directly engaged its program recipients and other stakeholders to receive feedback on how to improve their programming. All the RDAs dedicate the same program staff member to work with the recipient throughout the project life cycle to maintain continuity for the recipient. ACOA and CEDQ are using client surveys to measure satisfaction level related to program delivery, as well as regular monitoring and reporting of performance on established client service standards. ACOA’s website identifies information to help potential applicants access and understand information about its programs. CEDQ has established a centralized client service office which is responsible for all aspects of client management, including receiving recipient satisfaction survey results, and communicating these results to the responsible regional office.

FedDev Ontario held stakeholder engagement sessions in the summer and fall of 2012 to obtain feedback in relation to its role, visibility, outreach, and programs. These and other consultations were also undertaken to support the Agency’s program renewal initiatives. At the time of the audit, FedDev Ontario did not have a standard process to formally manage recipient feedback. An interim evaluation of the Agency recommended the use of recipient surveys to assess the extent to which program objectives have been achieved. FedDev Ontario is addressing opportunities to improve its management of stakeholder relationships and program performance. For example, the Agency is working with other RDAs and the Treasury Board Centre of Expertise for Grants and Contributions to identify opportunities to collaborate on a shared client relationship management system. Program service standards are currently being developed.

Establishing formal processes and data collection systems to receive client feedback helps to support the evaluation of program performance, program improvements and decision-making in relation to the future directions of programs.  

Overall, all three RDAs have engaged with their stakeholders to receive their feedback to improve program performance.

Conclusion

Overall, the management control frameworks over monitoring and reporting established and implemented by ACOA, CEDQ and FedDev Ontario support compliance with the Policy on Transfer Payments and the Directive on Transfer Payments.

Management Response

The findings and recommendations of this audit were presented to ACOA, CEDQ and FedDev Ontario management.

Management has agreed with the findings included in this report and will take action to address all recommendations applicable to their RDA.

Appendix A: Lines of Enquiry and Audit Criteria

Line of Enquiry Criteria

1. The RDAs have established program governance, risk management and internal controls processes to monitor recipient compliance and support RDA-wide monitoring and reporting requirements.

1.1: The RDAs have established, with consideration of risks and cost-effectiveness, oversight mechanisms, internal controls and performance reporting systems that support monitoring and reporting of its programs.

1.2: The RDAs have defined and communicated policies, procedures, controls and clear roles and responsibilities that govern the program’s monitoring and reporting processes.

1.3: The RDAs have identified monitoring and reporting risks for both the program and recipients and have developed a strategy to manage these risks.

2. RDA program monitoring and reporting requirements are developed and managed in consideration of program risk, results-management, and compliance with the Policy on Transfer Payments, its related Directive and applicable legislation.

2.1: The RDAs have designed a performance measurement strategy including performance measures, indicators, supporting data requirements, and a data collection strategy to support results reporting in relation to program objectives.

2.2: The RDAs have established a methodology to assign monitoring and reporting requirements to recipients on a basis that is proportionate to the recipient / project risk assessment rating.

2.3: The RDAs are monitoring the receipt of required deliverables from program recipients to ensure that they are in accordance with the risk assessment rating and program / funding agreement terms and conditions.

2.4: The RDAs review required deliverables to ensure the recipient has complied with the program / funding agreement terms and conditions.

3. RDA payments to recipients and reimbursements made to the RDAs are monitored to ensure they are performed in a manner that is compliant with program / funding agreement terms and conditions, the Policy on Transfer Payments, its related Directive and the Financial Administration Act.

3.1: RDA monitoring mechanisms are in place to ensure that payments to recipients are made in a manner that is compliant with established program / funding agreement terms and conditions, RDA practices, controls, service standards, the Policy on Transfer Payments, its related Directive and the Financial Administration Act.

3.2: Recipient reimbursements to the RDA are made in accordance with program / funding agreement terms and conditions, established RDA practices, and controls, the Policy on Transfer Payments, its related Directive, and the Financial Administration Act.

3.3: Funding agreement closure procedures are conducted as required.

4. Relationship management with recipients and continuous program improvement are RDA priorities.

4.1: The RDAs have established client relationship management mechanisms to engage with its recipients to achieve intended results and to measure performance and in pursuit of continuous program improvement.

Appendix B: Recommendations by Regional Development Agency and Priority Ranking

Recommendations ACOA CEDQ FedDev Ontario Priority Ranking (High, Medium or Low)

Table 2 Notes

Legend
A: Recommendation applicable.
N/A: Recommendation not applicable.

Recommendation 1

ACOA and FedDev Ontario management should take steps to ensure evidence of review of recipient progress reporting requirements is consistently documented.

A N/A A Medium

Recommendation 2

ACOA and CEDQ management should ensure that required payment verification and approval activities are performed and documented consistently with expected procedures and controls. ACOA and CEDQ should also take appropriate actions to recover the ineligible amounts paid to recipients.

A A N/A Medium

Appendix C: Selected Regional Development Agency Contribution Programs

Atlantic Canada Opportunities Agency (ACOA) – Business Development Program (BDP)

Focusing on small and medium-sized enterprises, the BDP provides access to capital in the form of interest-free, unsecured, repayable contributions. Not-for-profit organizations that provide business community support may also qualify for non-repayable assistanceFootnote 2. In 2011–12, the BDP invested $31.8 million in innovation projects, with over $11.8 million for technology adaptation.Footnote 3

Economic Development Agency for the Region of Quebec (CEDQ) – Quebec Economic Development Program (QEDP)

The QEDP helps support entrepreneurship, business performance, regional mobilization and investment while providing support for economic activity in Quebec communities to stabilize or strengthen their economies.Footnote 4 In fiscal year 2012–13, planned spending for the QEDP and other CEDQ programs was estimated at $208.6 million.Footnote 5

Federal Economic Development Agency for Southern Ontario (FedDev Ontario) – Southern Ontario Development Program (SODP)

FedDev Ontario was established in to help the southern Ontario economy mitigate and overcome regional and global economic challenges.Footnote 6 The SODP is FedDev Ontario’s core program that supports projects to stimulate local economies, business and communities to enhance their growth and competitiveness.Footnote 7In 2010–11, seven Southern Ontario Advantage Initiatives (SOAIs) were launched under the SODP to help the region's businesses and communities become more competitive, innovative and diversifiedFootnote 8. In 2011–12, the Agency expended $156.7 million through the SOAIs.Footnote 9 The Agency was renewed for an additional five years through Economic Action Plan 2013.Footnote 10

Page details

Date modified: