Skip to content Skip to institutional links

Common menu bar links

Breadcrumb Trail

Institutional links

Reports

Other Related Reports

Versions

Print Version

ARCHIVED - Evaluation of the Treasury Board Submission Process

Warning This page has been archived.

Archived Content

Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.

Appendix A: Logic Model

Appendix A: Logic Model - Text versions

Appendix B: Treasury Board Submission Process Evaluation Matrix

**Relevance**
Evaluation Question	Individual Number	Indicators	Focus Group: Assistant Secretaries	Interviews: TBS Analysts / Internal Contacts (n=10)	Interviews: Federal Organizations (n=12)	Interviews: External Stakeholders ( n=5)	Literature Review
1.0 What need does the Treasury Board (TB) submission process fulfill?	1.1	Rationale for TB submission process (FAA sections 7-11, TBS RPP, Guide to Preparing TB Submissions)		Legal only		1	1
	1.2	Alignment with relevant legislative requirements					1
	1.3	Comments and rationale for any changes in what is required to go to TB (e.g. changes in delegation of authority, changes in number of submissions)	1	1	1		1
	1.4	Appropriateness of what is required to be submitted (TBS reached an appropriate balance, e.g. regarding different types of submissions, different transactions)	1	1	1

**Results - Immediate Impacts**
Evaluation Question	Individual Number	Indicators	Focus Group: Assistant Secretaries	Interviews: TBS Analysts / Internal Contacts (n=10)	Interviews: Federal Organizations (n=12)	Web-Based Survey: Federal Organizations	Web-Based Survey: TBS Analysts	Web-Based Survey: TBS Internal Contacts	Data Analysis	Literature Review
2.0 Do federal organizations have an increased understanding of the elements of TB submissions, policies, and process?	2.1	Usefulness of guides, tools, and outreach provided by TBS			1	1				1
3.0 Do TBS employees have an increased understanding of the elements of TB submissions and policies?	3.1	Usefulness of guides, tools, and internal courses		1			1	1		1
4.0 Is TBS offering services that enable federal organizations' draft submissions to comply with TB authorities, policies, and directions?	4.1	Accuracy/consistency of TBS's pre-submission guidance		1	1	1	1
	4.2	Appropriateness of TB submissions being put forward		1			1	1
	4.3	Comparison of understanding of time required for TBS to review a submission				1	1	1
	4.4	Timeliness - Number of late submissions							1
	4.5	Relationship of TBS analysts to federal organizations (Change Agenda: strategic partner and trusted advisor, proactive risk manager, overseer vs. enabler, challenge vs. facilitation)		1	1	1	1	1
5.0 Does TBS's submission review process ensure that TB submissions comply with government authorities and policies?	5.1	Accuracy, consistency, usefulness, and timeliness of TBS advice/consultation during submission process		1	1	1	1
	5.2	Adequacy of time provided for input into TB submission documents					1	1
	5.3	Confidence that input from TBS consultations is reflected in final TB submission documents	1	1	1		1	1
	5.4	Limited legal contestation of the submission	1	Legal only			1
6.0 Is the TB submission process risk-based?	6.1	Extent of knowledge of existence of risk criteria		1	1	1	1	1
	6.2	Appropriateness of risk criteria (government-wide perspective, reflect ministers' concerns, reflect financial and non-financial risks)	1	1	1
	6.3	Extent/consistency of use of criteria		1
	6.4	Extent that TB submission process, due diligence, pr�cis, and decisions are impacted by risk rating	1	1
	6.5	Appropriateness of level of involvement in risk assessments (TBS vs. federal organizations)		1	1
7. 0 Are TB decisions well-informed and consistent with advice, guidance, and recommendations provided by TBS analysts?	7.1	Consistency of decision with recommendations, number and frequency of additional conditions applied by TB, number and frequency of challenges raised	1				1		1
	7.2	Number of deferrals and requests for more information							1
	7.3	Confidence in recommendations provided (e.g. with respect to risk, cost implications for government)	1						1
8. 0 Are mechanisms in place to ensure TB decisions are carried out?	8.1	Existence/adequacy of condition tracking processes: in federal organizations at TBS		1		1	1	1
	8.2	Timeliness/completeness of decision communication				1	1	1
	8.3	Completeness/usability of TB submission filing system		1			1	1

**Results - Intermediate/Long-Term Impacts**
Evaluation Question	Individual Number	Indicators	Focus Group: Assistant Secretaries	Interviews: TBS Analysts / Internal Contacts (n=10)	Interviews: Federal Organizations (n=12)	Interviews: External Stakeholders ( n=5)	Web-Based Survey: Federal Organizations	Web-Based Survey: TBS Analysts	Web-Based Survey: TBS Internal Contacts	Data Analysis	Literature Review
9.0 What is the level of quality of TB submissions?	9.1	MAF score element 5								1
9.0 What is the level of quality of TB submissions?	9.2	Completeness of TB submission documents: follow Guide to Preparing TB Submissions risks and mitigation strategies identified with sufficient information to allow for TBS analysis of risk asking for the right authorities appropriate level of justifications internal consultations in federal organizations are complete accuracy (tables are correct) clarity of TB submission	1	1	1		1	1	1
10.0 Does the TB submission process contribute to ensuring departmental/government management, programs, and spending are aligned with Government of Canada priorities?	10.1	Documented accounts of TB submissions contributing to alignment (e.g. Auditor General reports, MAF, Results for Canadians, TBS DPR)								1	1
	10.2	Explanations of whether or not TB submission process contributes to ensuring alignment	1			1
11.0 Are there any factors that are impeding the achievement of results?	11.1	Description of factors impeding achievement of results		1	1
12.0 Have there been any unexpected outcomes?	12.1	Description of unexpected outcomes		1	1

**Effectiveness**
Evaluation Question	Individual Number	Indicators	Focus Group: Assistant Secretaries	Interviews: TBS Analysts / Internal Contacts (n=10)	Interviews: Federal Organizations (n=12)	Interviews: External Stakeholders ( n=5)	Web-Based Survey: Federal Organizations	Web-Based Survey: TBS Analysts	Web-Based Survey: TBS Internal Contacts	Data Analysis	Literature Review
13.0 What is the level of resource allocation to submission process?	13.1	Changes in level of TB submission effort 5 years ago relative to today					1	1	1
	13.2	Trend analysis of number of submissions								1
	13.3	Estimates of time spent: on pre-submission stage reviewing/consulting on submissions briefing and preparing briefing documents (e.g. pr�cis) and other TB submission-related activities						1	1
	13.4	Identify rate, determine step and where resource increases would have most impact on process flow		1	1	1		1	1	1
	13.5	Factors impeding analysts' ability to contribute to submission process						1	1
14.0 Is the process efficient? What could be done to make the process more efficient?	14.1	Usefulness of measures taken as a result of the 2003 audit and the 2007 EXCO Retreat (Roadmap for enabling change)		1	1		1	1	1		1
	14.2	Identification of any measures taken by TBS to manage workload (e.g. risk-based approach to submission review, delegation of authority)			1	1	1	1	1
	14.3	Identification of any areas where efficiency can be improved	1	1	1	1	1	1	1
15.0 Are Canadians getting value for their tax dollars?	15.1	Analysis of efficiency, effectiveness, and impact indicators