This page has been archived.
Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.
Our approach to the collection of information for this evaluation relied upon four inter-related lines of enquiry, as shown in Exhibit II-1.
Exhibit II-1
Approach to data collection, analysis and reporting
This approach was designed to provide information from multiple sources to enable the evaluation issues to be assessed from several perspectives and to better understand the positions advanced by participants who are most closely involved with the use of foundations for public policy purposes. The study also had to be completed within a relatively short time period – from September, 2006, to January, 2007 – which necessitated a concentrated approach to data collection.
1. Documents review
Our evaluation team reviewed a broad range of documentation on the government's use of foundations to achieve policy goals, the evolution of the terms and conditions under which foundation funding has been provided, and the results achieved by various foundations. The principal sources of information included:
The review of documentation was used to identify information relating to the rationale for establishing foundations as instruments of public policy, governance and accountability requirements, results achieved to date, and characteristics of disbursements and operating costs.
2. Case studies
A series of six case studies of selected foundations were used as one of the two core data collection and analysis methods in the evaluation. The six case study foundations were:
The case studies were used to obtain insights into the appropriateness, effectiveness and costs of specific foundations, which were used, in conjunction with findings from interviews with other foundations and stakeholders, to identify common characteristics, themes and conclusions applicable to all, or most, foundations.
The methodology for the case studies involved:
The case study foundations were selected on the basis of four primary criteria:
In addition, the mix and balance of the case study foundations was also checked against three secondary criteria to ensure the six selected provided a reasonable representation of the full range of foundations in operation. These secondary criteria were:
Exhibit II-1 provides a summary of the characteristics of the six case study foundations against the above criteria.
Exhibit II-1
Characteristics of the case study foundations
Case Study Foundations |
Primary Criteria |
Qualifying Criteria |
||||||
Type of Funding Agreement |
Scale of Funding |
Sector Orientation |
Form of Funding |
Legal Basis |
Depart- ment |
Year |
# of Grants |
|
Canada Foundation for Innovation |
Fixed Term |
Large |
Infra- structure/ Tech. Demo. |
Project Grants |
Legislation |
Industry |
96/97 |
5 |
Millennium Scholarship Foundation |
Fixed Term |
Large |
Education |
Bursaries, Scholar-ships |
Legislation |
HRSD |
97/98 |
1 |
Genome Canada |
Fixed Term |
Large |
R&D |
Research Grants |
CCA1 |
Industry |
99/00 |
4 |
Aboriginal Healing Foundation |
Fixed Term |
Medium |
Community- Based Initiatives |
Project Grants |
CCA1 |
IRSRC |
97/98 |
2 |
Green Municipal |
Perpetual Endowment |
Large |
Infra- structure/ Tech. Demo. |
Project Grants & Loans |
CCA2 |
NRCan & Environment |
99/00 |
3 |
Pacific Salmon Endowment Fund Society |
Perpetual Endowment |
Small |
Community- Based Initiatives |
Project Grants |
B.C. Society Act |
Fisheries & Oceans |
00/01 |
1 |
1. I corporated under the Canada Corporations Act as a not-for-profit organisation.
2. Parent organisation, Federation of Canadian Municipalities, incorporated under the Canada Corporations Act.
3 Key informant interviews
Additional interviews were conducted with representatives of other foundations, their funding departments, central agencies and selected programs operating in similar fields to foundations. These interviews were used to obtain breadth of coverage regarding the appropriateness of using foundations, their effectiveness, and approaches to the design of governance structures and administration of funding agreements. A total of 26 interviews with 38 participants were conducted, involving:
A list of the participants in the interviewing program is presented in Appendix A.
4. Literature review
In preparation for the evaluation, TBS commissioned a study to identify international best practices pertaining to the governance and accountability of organizations that shared many features of the foundation model, relating to what have become known as "semi-autonomous organizations" and "non-departmental public bodies" (or "quangos"). We reviewed the findings from this research and conducted a limited further search of the academic and "grey" literature (non-peer reviewed, usually government-sponsored reviews) to determine any lessons regarding the effectiveness of these approaches and their governance. A list of the key documents reviewed is presented in Appendix B.
This evaluation study of the use of foundations draws on the findings from a series of cases studies; interviews with representatives of foundations, funding departments, central agencies and a small number of other stakeholders; and a supporting review of documentation and research literature.
In reviewing the findings from the work a number of limitations of the methodology should be borne in mind. Firstly, the participants in the key informant interviews formed a convenience sample, composed of people with direct roles in the management and monitoring of foundations, and their relationships with, and accountabilities to, the federal government. At a time when a number of the foundations are approaching periods where decisions regarding renewal and refunding, or winding up, will need to be made we had to be cognizant of the different degrees of possible self-interest at play and to seek to balance or cross-validate respective views.
Our sample of key informants had broad representation from across the spectrum of foundations and funding departments, which provided us with a good balance of perspectives on the appropriateness of foundations as instruments of public policy. However, the limited time within which the data collection and analysis was undertaken precluded us from including a larger pool of representatives of foundations' partners and stakeholders, particularly at other levels of government and within the public research community, who could have added to the richness of the analysis of effectiveness.
With regard to the analysis of what foundations cost, we had originally anticipated developing estimates of the costs of delivering foundations programs using departmental programs. However, we found that the very reasons why foundations have been established meant that undertaking these same activities within the departmental context would require a significantly different program design and delivery structure from that which foundations are able to use. In the time available, we were not able to develop a reliable basis for such comparisons and limited our analysis of costs to the cost structures and trends of foundations, particularly those of the case study foundations, and comparisons to the operating and administration costs of a number of similar government organisations. These comparators were chosen on the basis of the similarity of their programs and activities to those of some foundations, particularly those that research or infrastructure projects. We also supplemented this with a qualitative analysis that drew upon the foundation and departmental representatives' views of the cost differences that would arise if foundation activities were to be delivered using departmental programs.