ARCHIVED - Evaluation of the Treasury Board Submission Process

• Previous Page
• Table of Contents
• Next Page

This page has been archived.

Appendix E: Interview Guides

TBS Analyst Interview Guide

(Note: this is an example of the TBS Analyst Interview Guide)

Government Consulting Services (GCS) has been engaged by the Treasury Board of Canada Secretariat (TBS) to conduct an evaluation of the Treasury Board (TB) submission process. The purpose of this evaluation is to examine if the process continues to be relevant, if the intended results of the process have been achieved, and whether it is cost-effective and any improvements could be made.

As part of the evaluation, GCS is conducting interviews with key stakeholders involved in the process. The goal of the interviews is examine how the process is being implemented, collect information to assess the success of the process, and identify possible process improvements.

The following questions will serve as a guide for our interview. In some cases, questions will not be relevant to your particular situation. The interviewers will focus on those questions relevant to you. Please note that the responses you provide will not be attributed to you in the evaluation report (only aggregate information will be released) or in any documentation provided to the responsible department. If you have any questions regarding the enclosed, please feel free to contact the GCS project manager, Susan Cole, at (613) 943-3401.

For the purpose of this evaluation, the TB submission process within TBS is defined by the following five steps.

1. Awareness: TBS provides awareness and education regarding the TB submission process to all stakeholders.
2. Guidance: TBS provides initial pre-submission guidance, planning, and strategies to federal organizations.
3. Analysis: TBS provides advice, consultation, and due diligence on draft submissions presented by federal organizations.
4. Recommendation: TBS writes the submission précis and provides advice to ministers at briefings and meetings of the Treasury Board.
5. Decision: TBS documents and communicates the decision and establishes the tracking of conditions.

The scope of the evaluation focuses only on "Part A" submissions and excludes "Part B" submissions.

Background

1. What is your role and involvement with the TB submission process?

1. How long have you been in this role?
2. How many submissions have you been involved with in the past year?
3. Do you distinguish between standard and complex submissions? If so, how? Are there any other ways that you categorize submissions for review?

Relevance

2. Based on your understanding, what are the TB submissions to be used for?

1. Are TB submissions submitted for purposes for which there are other more appropriate vehicles?
2. Have you noticed any differences since the implementation of the Change Agenda in the areas for which TB submissions are used? Please explain.

Impacts

3. Do TBS analysts have a strong understanding of the elements of a TB submission, the policies and guidance related to TB submissions, and the TB submission process? If not, what would help improve their level of understanding?

4. In the past year, have you consulted any of the following material regarding the TB submission process? Did it help improve your understanding of the elements of TB submissions, policies, and TB submission process?

1. A Guide to Preparing Treasury Board Submissions
2. Other TBS tools (please describe the tool)
3. Courses (Canada School of Public Service TB submission course, TBS Analyst Boot Camp, etc.)

5. Are TBS analysts providing accurate, consistent, useful, and timely policy advice/interpretation in regards to TB submissions:

1. At the pre-submission stage (when federal organizations are trying to decide whether a TB submission is required)?
2. During the TB submission review stage?
3. Is the advice consistent within your sector? Among program and policy sectors?
4. Is the advice consistent across TBS?

6. In general, do TBS analysts have the appropriate training and support to enable them to provide accurate, consistent, useful, and timely advice to federal organizations in regards to the TB submission process? If not, what is required?

7.

1. In the past year, have federal organizations shared drafts for consideration/review that did not require a TB submission?
   1. What percentage of drafts did you receive that fell into this category?
   2. Did the authors consult TBS prior to sharing?
   3. Has this been getting more or less frequent?
2. In the past year, have federal organizations submitted drafts for approval that did not require a TB submission?
   1. What percentage of drafts did you receive that fell into this category?
   2. Did the authors consult TBS prior to submitting?
   3. Has this been getting more or less frequent?

8. In the past year, how frequently have federal organizations sought exemptions and exclusions with their TB submissions? Please explain.

9. How would you characterize your relationship with the individuals in federal organizations who you interact with in regards to the TB submission process?

10. Does your program area have a review process in place to validate the advice provided by its analysts? If yes, please briefly describe this process.

1. Do you participate in this process? If yes, in what role?

11. Do you have confidence that the advice and consultation you provide is reflected in the final TB submission documents? Please explain.

12. Are you aware of the risk criteria applied to TB submissions?

13. Please describe the role of each stakeholder (federal organization and TBS) in the risk assessment process?

14. Do the risk criteria appropriately define the risks associated with TB submissions?

1. Are they reflective of a government-wide perspective, ministers' concerns, financial and non-financial risks, etc.?

15. Are the risk criteria being applied consistently across TB submissions? Please explain.

16. In your view, in the past year, how has the risk rating associated with a submission influenced your program or policy division's response to it with regards to the process, due diligence, précis, etc.?

17. Has the quality of the submission documents being put forward to Treasury Board changed over the past five years? If yes, how so?

18. Are there mechanisms in place to ensure that TB decisions are carried out?

19. What is the process for tracking the conditions applied to approved submissions?

1. To what extent is this process effective?

20. How are submission approvals filed for future reference?

1. To what extent is the filing system useful and complete?

21. Have there been any unexpected benefits or challenges related to the process? Please explain.

Cost-Effectiveness and Potential Alternatives

22. Is the submission process efficient?

1. Has TBS undertaken any measures to manage the TB submission workload within TBS or to improve the efficiency of the process?
2. What additional measures, if any, could make it more efficient?
3. Are there any specific steps along the submission process where the flow of work gets stalled (i.e. bottlenecked)? If so, do you have any suggestions for improvement?

23. Are you aware of the measures proposed by internal studies such as the 2003 internal audit of the TB submission process and the 2007 EXCO Retreat? Did you find that they helped to improve the efficiency or effectiveness of the program?

Some measures undertaken as a result of the 2003 audit include:

1. Security measures with respect to transferring a submission - i.e. new procedures, more secure facsimile machines, and more secure couriers
2. New timelines - Are the new timelines reasonable and allow for an efficient process?
3. Definition of classification - i.e. new definitions have provided clarity and simplified the decision-making process
4. Measures to address analyst turnover and corporate memory - i.e. new measures in place to help new analysts adapt to the work and ensure that when analysts leave, the knowledge is transferred to current staff

Some measures taken as a result of the 2007 EXCO Retreat include:

1. Ministerial renewal of terms and conditions - Proposed authorization for ministers to extend expiring terms and conditions for up to three years within certain parameters, which include strategic reviews, evaluations, and audits
2. Increasing departmental delegated authorities - Program sectors review delegations for departments to determine if adjustments are warranted
3. Chief financial officer (CFO) sign-off - CFO or delegate to sign off on all draft TB submissions before TBS review to signal appropriate quality and completeness
4. Managing low-risk submissions differently - Flag and process TB submissions based on combined risk rating
5. Altering the submission calendar - Emphasize prerogative of TBS to determine scheduling of submissions, including in Guide to Preparing Treasury Board Submissions
6. Streamlining supply processes - Program sector assistant secretaries and Expenditure Management Sector are working together to review Estimates processes, looking at each activity through lenses of accountability, duplication, and value added, including options to reduce the overall burden of preparatory work for Senate briefings

24. Do you have anything else to add?

LEGAL ONLY

25. Has the amount of legal contestation of TB submissions changed over the last few years? If so, how?

Federal Organization Interview Guide

(Note: this is an example of the Federal Organization Interview Guide)

Government Consulting Services (GCS) has been engaged by the Treasury Board of Canada Secretariat (TBS) to conduct an evaluation of the Treasury Board (TB) submission process. The purpose of this evaluation is to examine if the process continues to be relevant, if the intended results of the process have been achieved, and whether it is cost-effective and any improvements could be made.

As part of the evaluation, GCS is conducting interviews with key stakeholders involved in the process. The goal of the interviews is examine how the process is being implemented, collect information to assess the success of the process, and identify possible process improvements.

The following questions will serve as a guide for our interview. In some cases, questions will not be relevant to your particular situation. The interviewers will focus on those questions relevant to you. Please note that the responses you provide will not be attributed to you in the evaluation report (only aggregate information will be released) or in any documentation provided to the responsible department. If you have any questions regarding the enclosed, please feel free to contact the GCS project manager, Susan Cole, at (613) 943-3401.

For the purpose of this evaluation, the TB submission process within TBS is defined by the following five steps:

1. Awareness: TBS provides awareness and education regarding the TB submission process to all stakeholders.
2. Guidance: TBS provides initial pre-submission guidance, planning, and strategies to federal organizations.
3. Analysis: TBS provides advice, consultation, and due diligence on draft submissions presented by federal organizations.
4. Recommendation: TBS writes the submission précis and provides advice to ministers at briefings and meetings of the Treasury Board.
5. Decision: TBS documents and communicates the decision and establishes the tracking of conditions.

The scope of the evaluation focuses only on "Part A" submissions and excludes "Part B" submissions.

Background

1. What is your role and involvement with the TB submission process?

1. How long have you been in this role?
2. How many submissions have you been involved with in the past year?

Relevance

2. Based on your understanding, what is the TB submission process to be used for? Are there other vehicles that might be more appropriate for these areas?

Impacts

3. In the past year, have you consulted any of the following material regarding the submission process? Did it help improve your understanding of the elements of TB submissions, the policies and guidance related to TB submissions, and the TB submission process?

1. A Guide to Preparing Treasury Board Submissions
2. Other TBS tools (Canada School of Public Service TB submission course, etc.)
3. TBS outreach (please describe)

4. Do the TBS analysts you have been in contact with have a strong understanding of the elements of TB submissions, the policies and guidance related to TB submissions, and the TB submission process? If not, what would help improve their level of understanding?

5. When deciding whether a TB submission is required, do you consult a TBS analyst? Why or why not?

1. If so, was the advice helpful?

6. In the past year, how frequently has your organization sought exemptions and exclusions through TB submissions?

7. How would you characterize your relationship with TBS and the analysts that you interact with at TBS?

8. Once you submit a draft to TBS, to what extent is the advice and consultation provided by TBS on the draft accurate, consistent, useful, and timely?

9. Does TBS Legal Services provide advice consistent with your own organization's Legal Counsel?

10. To what extent do you incorporate the input TBS provides into the final version of the submission?

11. Are you aware of the risk criteria applied to TB submissions?

12. Please describe the role of each stakeholder (federal organizations and TBS) in the risk assessment process?

13. Do the risk criteria appropriately define the risks associated with TB submissions?

1. Are they reflective of a government-wide perspective, ministers' concerns, financial and non-financial risks, etc.?

14. Has the quality of the submissions your organization puts forward to Treasury Board changed over the past five years?

1. If so, what has contributed to the change?
2. Does this show in your organization's MAF assessment?

15. Have there been any unexpected benefits or challenges related to the process? Please explain.

Cost-Effectiveness and Potential Alternatives

16. Is the submission process efficient?

1. What additional measures, if any, could make it more efficient?
2. Are there any specific steps along the submission process where the flow of work gets stalled (i.e. bottlenecked)? If so, do you have any suggestions for improvement?

17. Do you have anything else to add?

External Stakeholder Interview Guide

(Note: this is an example of the External Stakeholder Interview Guide)

Government Consulting Services (GCS) has been engaged by the Treasury Board of Canada Secretariat (TBS) to conduct an evaluation of the Treasury Board (TB) submission process. The purpose of this evaluation is to examine if the process continues to be relevant, if the intended results of the process been achieved, and whether it is cost-effective and any improvements could be made.

As part of the evaluation, GCS is conducting interviews with key stakeholders involved in the process. The goal of the interviews is examine how the process is being implemented, collect information to assess the success of the process, and identify possible process improvements.

The following questions will serve as a guide for our interview. In some cases, questions will not be relevant to your particular situation. The interviewers will focus on those questions relevant to you. Please note that the responses you provide will not be attributed to you in the evaluation report (only aggregate information will be released) or in any documentation provided to the responsible department. If you have any questions regarding the enclosed, please feel free to contact the GCS project manager, Susan Cole, at (613) 943-3401.

For the purpose of this evaluation, the TB submission process within TBS is defined by the following five steps:

1. Awareness: TBS provides awareness and education regarding the TB submission process to all stakeholders.
2. Guidance: TBS provides initial pre-submission guidance, planning, and strategies to federal organizations.
3. Analysis: TBS provides advice, consultation, and due diligence on draft submissions presented by federal organizations.
4. Recommendation: TBS writes the submission précis and provides advice to ministers at briefings and meetings of the Treasury Board.
5. Decision: TBS documents and communicates the decision and establishes the tracking of conditions.

The scope of the evaluation focuses only on "Part A" submissions and excludes "Part B" submissions.

Background

1. What is your familiarity with the TB submission process?

1. What kind of involvement have you had with the process (prompt - do they write submissions or have they been consulted on them by federal organizations)?

Relevance

2. Based on your understanding, what is the TB submission process to be used for? Are there other vehicles that might be more appropriate for these areas?

Impacts

3. Does the TB submission process contribute to ensuring that departmental/government management, programs, and spending are aligned with Government of Canada priorities? Please explain.

4. Are there any particular benefits or challenges related to the process?

Cost-Effectiveness and Potential Alternatives

5. Is the submission process efficient?

1. What additional measures, if any, could make it more efficient?
2. Are there any specific steps along the submission process where the flow of work gets stalled (i.e. bottlenecked)? If so, do you have any suggestions for improvement?

6. Do you have anything else to add?

• Previous Page
• Table of Contents
• Next Page