ARCHIVED - An Enhanced Framework for the Management of Information Technology Projects - Part II Solutions: Putting the Principles to Work

This page has been archived.

An Enhanced Framework for the Management of Information Technology Projects

An Enhanced Framework for the Management of Information Technology Projects - Part II Solutions: Putting the Principles to Work

March 1998
Chief Information Officer Branch
Treasury Board of Canada Secretariat

1. Executive Summary

This document is a companion to the document entitled: The Enhanced Framework for the Management of Information Technology Projects (Enhanced Framework) approved and published in May 1996. Treasury Board has directed departments to apply this framework to existing projects, as applicable, and to all future information technology projects.

Implementation of the Enhanced Framework is designed to ensure that government information technology projects fully meet the needs of the business functions they are intended to support, deliver all expected benefits and are completed on time, to cost and functionality. It is also anticipated that implementation will, over time, enable departments to increase their levels of authority.

Departments are required to attest compliance to the Enhanced Framework when seeking Treasury Board project approvals.

1.1 Purpose of the Document

The purpose of this document is to facilitate implementation of the Enhanced Framework within federal government departments.

• It provides an overview of the Enhanced Framework, the principles and related practices;
• It identifies where and how to begin the process of implementation. A model, entitled the Departmental Improvement Plateaux, provides departments with a four-step approach. Each step contains target activities and target dates for implementation. It is suggested that Plateau 0 be completed by March, 1998;
• It outlines solutions to assist departments in applying the Framework. It is expected that departments will introduce some of these solutions during the project initiation phase, Plateau 0. These solutions can and should be tailored for the specific departmental environment;
• It describes the roles and responsibilities of the key players in project delivery. The key departmental players are the Departmental Executive Committee, Business and Information Technology/Information Management (IT/IM) Senior Management, Program and Operations Managers, Corporate Services, Audit, the Project Team and the Project Governance Team. Key external players include PWGSC and the Treasury Board Secretariat.
• It provides guidance on how to get started.

The document is descriptive in nature. Each department is expected to define its own implementation strategy and plan based upon its own experiences, expertise and process improvement initiatives to date.

1.2 Chaos⁽¹⁾ No More - The New Target Environment

The Enhanced Framework addresses the establishment of sound management and software engineering processes and best practices. Implementation of these processes and practices will deliver the following results:

• IT Projects that clearly support departmental business directions and priorities;
• Effective client partnerships where clients are integral to all project phases;
• a management framework that balances the government's corporate requirements with departmental empowerment needs;
• an explicit governance structure;
• a cadre of information technology project managers who are trained in the application of the Enhanced Framework and designated common tools and methods;
• a consistent project manager discipline;
• project teams with the understanding and a commitment to the Enhanced Framework and the necessary competencies to deliver the project;
• established effective mechanisms for monitoring the status and assessing the true performance of projects;
• projects structured to ensure risks are identified and managed;
• a set of best practices, processes; and
• a continuous learning culture, with departments sharing knowledge and experience.

Full implementation of the Enhanced Framework will result in the environment described by the statements above. This is the target and achieving it will require belief in the possibilities, diligent and sound management practices, dedication in terms of time and resources, and perseverance. Senior Management must play a facilitative and active role, enabling the creation of this climate.

This new environment will permit departments to deliver IT/IM projects consistently and successfully.

1.3 Intended Audience

This document is intended for two primary audiences:

• IT/IM Senior Management whose role is to initiate improvement and to facilitate the development the Enhanced Framework environment through the provision of leadership and resourcesand
• Senior Business Management whose role is to actively support and participate in the improvement initiatives.

Together, these are the enablers of the changes espoused in the Enhanced Framework.

Three other important audiences have a crucial role to play:

• The Business Project Teams and providers of IT/IM products and services whose role is to actually create the environment and use elements of the Enhanced Framework;
• The Departmental CIO Branch and the CIO whose role is to communicate and promote the concepts of the Enhanced Framework and to assess departmental compliance with it;
• The Departmental Executive Management Team (DM & ADMs) whose role is to explicitly endorse and support the Enhanced Framework.

1.4 Relationship to Other Documents

This document and the Enhanced Framework document support and complement existing policies and regulations of the federal government regarding the management and delivery of  IT/IM projects. These are:

Administrative Management - Capital Plans, Projects and Procurement

• Project Approval
• Project Management
• Management of Major Crown Projects
• Policy on Selecting Preferred Procurement Strategies

Administrative Management – Real Property, Material and Risk Management

• Risk Management Policy

Information Management

• Strategic Direction for Government: Information Management
• Enhancing Services through the Innovative Use of Information Technology
• Management of Information Technology
• Guidelines

Additional supporting guidelines can be found in

• Treasury Board Information Technology Standards (supplementary) –TBITS 27
• Report of the Independent Review Panel on Modernization of Comptrollership in the Government of Canada

2. An Overview of the Enhanced Framework

The government is committed to delivering its programs and services more efficiently and effectively through the use of information technology. The Project Management Office (PMO) was formed under the auspices of the Treasury Board to provide guidance to departments such that government IT projects:

• satisfy the requirements of the program functions or services they are designed to support;
• deliver all the expected benefits; and
• are completed on time and within budget.

2.1 Principles of the Enhanced Framework

The PMO, in conjunction with other federal departments, published The Enhanced Framework for the Management of Information Technology Projects in May 1996. While remaining within the existing Treasury Board policy framework, it provides guidance for future improvements to the IT project management regimes currently found within departments.

This document defined four principles that set the broad parameters within which information technology projects are to be managed.

Principle #1: Projects are aligned with and support the business directions.

Information technology projects are undertaken to achieve successful and economical support of a business function. The Project Sponsor⁽²⁾, Project Leader⁽³⁾, and Project Manager⁽⁴⁾, together with the management and major users of the business function, must ensure that the project achieves these goals and delivers the expected benefits.

Principle #2: Clear accountabilities are established

Information technology projects can be complex undertakings. The responsibilities of all parties must be clearly defined and their delegated authority specified. Problems must be resolved in a timely manner to prevent them from threatening the success of the project and the achievement of the expected benefits.

Principle #3: Project managers are developed in, and work within, a corporate discipline

The project manager is the key for the successful completion of the project and the achievement of the expected benefits. The project manager, therefore, should have the appropriate training, skills and experience to manage the scope and risks of the stated project. A corporate approach to the development or acquisition of project managers based on government-wide practices, common tools and methodologies is required.

Principle #4: Project management decisions are based on risk management

Typically the government has emphasized meeting the target date of projects, with the result departments have pressed ahead to meet commitments, despite indications of serious problems or risks. Risk management provides a disciplined environment for proactive decision making to:

• continually assess what could go wrong;
• determine which risks must be dealt with; and
• implement strategies to deal with those risks

Projects must be planned, organized and structured to ensure success from their initial organization and planning through design, development and implementation.

Too often attention has been focused on the visible project with its attendant issues, while the root causes of the problems have remained hidden within the organization surrounding the project environment. Thus the order and wording of the Principles was chosen with great care. Each forms a part of the foundation for project success upon which the next is built. All are crucial in creating the environment within which a project can be expected to succeed.

The figure below illustrates this idea of foundation and supporting structure and shows the four principles in relation to one another.

Figure 1: The Enhanced Framework Pyramid

2.2 Practices of the Enhanced Framework

In the past, project success has relied on the heroic efforts of the project team. Often the team has had to work in isolation, unsupported or misunderstood by the larger IT or Business organizations. Project success has been dependent upon the creativity, determination and relentless hard work of the project team. The Enhanced Framework Pyramid illustrates that several layers of support must be in place for a project to be successful. Implementation of all layers of the Pyramid will ensure that projects are created within and treated as an integral part of organizational processes.

Each layer has a number of associated supporting management practices.

The Business Directions (Principle 1) of a department provide the foundation for project success. This layer includes such practices as:

• projects being compatible with both business and information management plans of a department;
• full business-case analyses being performed; and
• full and complete involvement of the client in all phases of the project.

Once a project has been defined that meets the Business Directions of a department, then the Accountabilities (Principle 2) for the project must be established. This layer ensures that the following practices are defined and communicated:

• the accountability of departments for projects;
• the role of TBS for the project framework;
• the contracting role of PWGSC;
• the specific and explicit accountabilities and responsibilities of the multiple stakeholders;
• the degree of organization readiness;
• the explicit roles, responsibilities and authorities of the Project Sponsor, Project Leader and Project Manager;
• the required competencies of the project team; and
• the assignment internally of core responsibilities and functions.

The third layer, Corporate Discipline (Principle 3), refers to the development of project managers (PMs) across government to ensure they have the requisite knowledge, skills and experience to manage the project's scope, complexity and risk profile. This layer consists of such practices as:

• a consistent project manager discipline;
• government-wide PM development;
• a continuous learning culture; and
• a PM support network.

These three layers: Business Directions, Accountabilities and Project Management Discipline are the organizational practices that must be in place better ensure project success. Within the project itself, Project Management Decisions (Principle 4) have the following practices associated with it:

• decisions are based on risk and projects are structured for manageable risk;
• project off-ramps are linked to a gating process;
• methodologies and tools are defined and used that allow for risk assessment, determination of project complexity, early product delivery, change management, measurement and metrics; and
• a complement of staff exist who are trained in the use of common methodologies and tools.

2.3 Expected Benefits of Framework Implementation

The Treasury Board has directed departments to apply this framework to existing information technology projects as applicable, as well as to all future IT/IM projects and to so attest when seeking Treasury Board project approvals. Departments are expected to apply elements of the framework in an intelligent fashion appropriate to the risk and size of each project. The anticipated benefits resulting from the effort of implementation are many.

Publicly declared and recorded benefits⁽⁵⁾ include:

• A more open and cooperative development environment between IT staff and client staff;
• A greater understanding and appreciation of the risks involved in software development projects and the necessity of managing these risks. Continuous risk management provides a disciplined environment for proactive decision making, for assessing continually what could go wrong, for determining which risks are important to deal with and for implementing strategies to deal with those risks.
• Increases in productivity, thus allowing more development with same staff complement;
• Decreases in the costs to deliver a project thus allowing development budgets to go further;
• Improved client satisfaction as a result of quality improvements and a reduction in calendar time to deliver projects;
• Reductions in defect and error rates;
• Schedule improvements, including a significant reduction in late projects; and
• Reductions in rework as a result of reduced root causes of errors.

Implementation of the Enhanced Framework will allow departments to accrue similar benefits and leverage their IT investments to increase the effectiveness and efficiency of government programs and services.

3. Where to Start

Since the publication of the Enhanced Framework in May of 1996, departments have not been idle. Pilots have been conducted, assessments have been made, lessons have been learned, and goals have been defined for the implementation of improvements within IT/IM organizations.

What follows is a recommended strategy that will allow departments to integrate these plans with the requirements of the Enhanced Framework. This strategy contains:

• a step-wise approach and timeframes for implementation;
• an outline of the activities at each stage;
• a brief description of the roles and responsibilities of the players required for successful implementation;
• a description of solutions/approaches/methods available from Treasury Board Secretariat that will be of assistance to departments during implementation; and
• a broad outline of the initial tasks required to get started.

3.1 Departmental Improvement Plateaux

It is understood that departments are not beginning implementation of the Enhanced Framework from a standing start. Some departments have already launched IT process improvement initiatives. The chart below, however, provides the larger government-wide view of implementation and the steps that departments are expected to have either underway or completed.

Figure 2 – Departmental Improvement Plateaux

Organizations cannot be improved overnight. These improvements require time, dedication and perseverance. Plateau 0 provides departments with the opportunity to immediately implement specified solutions within projects that will increase the likelihood for their success. It also gives departments the impetus to plan the strategy, tactics and implementation approach for the next four Plateaux.

Plateaux 1 and 2 address project processes that must exist for practices to be repeatable and project outcomes to be predictable. These processes must be in place and effectively performed if project teams are expected to deliver quality products.

Plateaux 3 and 4 address the broader organizational processes that must exist for projects to be consistently successful over the longer term. Improvements to key areas are cyclical in nature, with improvements planned, implemented, reviewed and then built upon.

Details of the goals to be met for Plateaux 0-4 can be found in Appendix 2

The Departmental Improvement Plateaux do not dictate the approach to be used by departments to improve but rather focuses on the concrete steps towards the ultimate goal of significantly improving the government's ability to deliver and manage IT projects. Developed independently from any methodology or improvement model, this path reflects the initial Enhanced Framework findings and the priorities of government departments.

Plateau 0: Baseline

This first Plateau is designed to ensure that departments have in place the basic project solutions required to initiate and manage a project. These include defining a clear and explicit business case, a procurement strategy, a project charter; a gating and review process; a project planning and control mechanism and a risk management approach.

The second objective of Plateau 0 is to ensure that departments create a plan for advancing to the next Plateau. Without a plan, it is unlikely that departments will make the improvement gains as identified in the Enhanced Framework. It is expected that by March of 1998, departments will have implemented the basic project solutions and created their plans for achieving their next Plateau.

Plateau 1: Project Focus

Project planning, tracking and oversight, and contract management (if applicable) begin once a project has been approved and initiated. While most departments already have processes in place that address these functions, the achievement of Plateau 1 will ensure that there are no gaps or weaknesses. The objective of this plateau is to achieve proper planning for projects in government departments and to establish sufficient visibility into actual progress to allow senior management to take effective action when the project deviates from plan. It is also in Plateau 1 that actions are taken to ensure project managers have the required knowledge and experience and the tools to support them and the project. Departments are to have implemented these improvements by March of 1999, and have a plan in place to move to their next Plateau.

Plateau 2: Product Focus

Plateau 2 seeks to establish, at a project level, consistent and effective controls and processes that when followed will ensure:

• changes to requirements follow a clearly defined effective change management process;
• product integrity is maintained throughout the life of the project; and
• product quality is acceptable within defined parameters.

Again, many departments have processes in place and achieving this Plateau will ensure that elimination of any gaps or weaknesses. Actions continue to be taken to ensure plans are in place for addressing Plateau 3 and that the project management training and education activities (professional development) continue.

Plateau 3: Organizational Effectiveness

The third Plateau deals with making the processes and practices established in Plateaux 0-2 the way that government departments do business for all their projects. The objective of this Plateau is to ensure that the best processes implemented in one project within a department are carried through to the other projects within the organization. Organization-wide issues such as the training of personnel and the documentation of processes are also addressed at this level.

Plateau 4: Continuous Improvement

The final Plateau deals with improving the organizational effectiveness of departments on a continuous basis when managing and delivering projects (e.g. deliver projects faster, better, cheaper). This Plateau includes quantitative techniques to measure processes, and proactive activities to streamline and improve these and other existing practices.

This path for improvement will guide Enhanced Framework implementation by establishing clear priorities, objectives and time frames.

3.2 Departmental Implementation Responsibilities

It is expected that all departments will be familiar with and committed to the implementation of the both the precepts and the solutions associated with the Enhanced Framework by March of 1998. Such a commitment will place the Enhanced Framework as a priority item on departmental action agendas.

The diagram below depicts those who have a major role to play in the implementation of the Enhanced Framework. This diagram does not depict every player in every circumstance. Rather it is an attempt to portray a generic situation. Departments are expected to adapt this portrayal to their own unique and individual circumstances.

All of the players depicted are key to the successful implementation of the Enhanced Framework. Failure on the part of any player to actively support and/or participate in its implementation will likely result in failure of the whole.

Figure 3: Key Departmental Participants in Implementation

Each group will support the implementation of the Enhanced Framework in the follow ways:

Departmental Executive Committees:

• Communicate the critical need for change
• Explicitly endorse and mandate the Enhanced Framework
• Assign responsibility and ensure compliance
• Harvest and leverage the benefits

Business Senior Management:

• Actively support Senior Management in their desire to implement the Enhanced Framework and communicate the need for change
• Assume the role of Project Sponsor
• Actively participate in the decision making process for business issues which affect the project
• Actively participate on the Project Governance Team
• Assign responsibility and resources for implementation of the Enhanced Framework elements that affect their environment
• Leverage and harvest the benefits of implementation

IT Senior Management Team

• Champion and commit to implementation of the Enhanced Framework
• Actively communicate the need for change and progress being made
• Assign resources
• Actively participate on the Project Governance Team
• Manage the boundaries between IT, the Business Units and Corporate Services

Project Governance Team

• Use the Enhanced Framework solutions tailored for their situation
• Ensure compliance with the Framework
• Communicate successes and 'best practices' developed
• Acquire new knowledge as appropriate
• Work together as a team in mutually supportive and explicitly defined roles
• Assist and advise project partners
• Liaise with the process team

Process Improvement Team

• Modify and tailor improvement processes to include Enhanced Framework
• Create the stimulus for improvement
• Set content and establish sponsorship
• Define the management process
• Design the measurement process
• Implement the change
• Gather metrics
• Demonstrate and publicize progress

Business Program Managers/ IT Operations Managers

• Understand the Enhanced Framework and apply as required
• Assign resources to the project and process teams
• Actively participate on the Project Governance Team as required

The Project Development Team

• Use the Enhanced Framework and it solutions
• Institute a program of risk management for the project
• Report progress accurately and timely

Audit and Review

• Audit IT projects against the precepts and concepts of the Enhanced Framework
• Provide feedback into the Framework to ensure that it continues to develop and reflect departmental needs
• Provide assurance on the adequacy and cost effectiveness of controls being built into the product

Corporate Services

• Modify policies, practices and procedures to reflect key concepts of the Enhanced Framework, where required
• Ensure contractual resources on the Project Development Team comply with the Framework
• Actively participate on the Project Governance Team when required
• Support the project through resourcing, procurement, finance, legal and security

3.3 Implementation Responsibilities for Key Supporting Participants

Key supporting participants are a diverse group whose role is to support the project, the department and/or each other in the implementation of the Enhanced Framework. For discussion purposes, the roles of these participants are treated as mutually exclusive. As well, no attempt has been made to identify every possible participant, as this will depend on the nature of the specific project. Instead, the implementation responsibilities of a representative sample are provided.

PWGSC (primarily Procurement: Science, Informatics and Professional Service Sector & Major Crown Projects Service Lines)

• Modify policies, practices and procedures to align with the Enhanced Framework
• Ensure contractual resources on the Project Development comply with the Enhanced Framework

PWGSC (The Institute)

• Modify training offerings to comply with the Enhanced Framework
• Schedule and deliver training for project sponsor, project leader and project manager
• Maintain an inventory of all project managers

TBS

• Align policies with the Enhanced Framework
• Apply the concepts of the Enhanced Framework against the submission process
• Facilitate implementation of the Enhanced Framework through strategic relationship building and the provision of support
• Provide increased levels of authority to departments where sustained compliance with the Framework has been demonstrated

Office of the Auditor General

• Audit IT projects against the precepts and concepts of the Enhanced Framework
• Recognize those departments who have implemented the Enhanced Framework and have learned from those experiences
• Provide feedback into the Framework to ensure that it continues to develop and reflect government needs

Canadian Centre for Management Development

• Build the Enhanced Framework into executive training
• Schedule the Project Sponsor and Project Leader symposia

4. Enhanced Framework Solutions

The goal of any IT/IM project is to deliver a product. This may be a new system, a modified or enhanced system, hardware, and/or any other supporting components. Implementation of the Enhanced Framework will enable project teams to focus on this goal.

In order to support and facilitate departmental implementation, the Enhanced Framework document (published May 1996) outlined an action plan calling for the development of concrete tools and techniques that could assist departments with implementation. As a result pilots have been conducted, assessments have been made, lessons have been learned, goals have been defined, priorities have been established and solutions have been developed. Implementing these solutions will complete the supporting foundations required to ensure project success.

These tools and techniques fall into one of three sets:

• solutions that directly support the project;
• solutions that improve both the primary processes within a project and other supporting processes external to the project (but related to the IT/IM organization); and
• solutions that support the people involved in project delivery.

These solution sets transcend the boundaries of the pyramid layers. These concepts are illustrated in Figure 4, on the following page.

 

Figure 4: Solution Sets

The solution sets contain templates, samples, strategies, tools, methodologies, standards, and/or programs.

• The Project Solution Set contains approaches that pertain to governance, procurement, risk management and the planning and control of a project. These will be of assistance for departments implementing Plateau 0, as defined in the Departmental Implementation Plateaux.

• Process improvement models, guides, assessment tools and methodologies are identified in the Process Solution Set. These will be of interest to those departments implementing Plateaus 1-4.

• Solutions that assist in the support and development of the people involved in a project are defined in the People Solution Set. This set reinforces and supports the other two sets described and will assist in the realization of Plateaux 0 – 4.

4.1 Overview of Solutions

The following chart illustrates the solutions being developed by the Treasury Board Secretariat.

Project Solution Set	Solutions
Governance Structure	Business Case Project Charter Gates, Reviews and Approvals
Procurement Reform	Benefit-driven Procurement
Risk Management	Continuous Risk Management Approach Selected Methods/ Tools/Techniques Lessons Learned database
Planning and Control	Tracking and Control Scenario Function Point Analysis Earned Value Measurement Benchmark database

 

Process Solution Set	Solutions
Process Improvement Models and Guides	Capability Maturity Models - Software Engineering Institute (SEE) Key Process Area Handbooks (SEI) EssentialSet Process Templates - Software Productivity Centre (SPC) IDEAL Guide to Process Improvement Tailoring Guidelines Lessons Learned database
Assessment Methodologies and Tools	Enhanced Framework Questionnaire CMM-Based Appraisal for Internal Process Improvement (SEI) S:PRIME (GrafP), Softguide (SPC) Capability Assessment Questionnaires

 

People Solution Set	Solutions
Project Management Support	Core Competencies Symposia Series Project Management Guide Project Management Tools Survey Web Site
PM Career Development	Accreditation Program Project Manager Network Education Program

These solution sets are designed to provide departments with a starting point. As with any tool, tailoring is required by the department to ensure that the tools directly support the specific environment.

4.2 Where and When to Use The Solutions

The chart below aligns the solutions defined with the appropriate implementation Plateau. The 'x' in the diagram indicates where a particular solution should be introduced or its use begun. Once a solution or tool has been initialized, its use is expected to be refined and continued through the remaining Plateaux. These solutions are described in further detail in the sections that follow.

Solutions (Project Solution Set)	Plateau 0	Plateau 1	Plateau 2	Plateau 3	Plateau 4
Business Case	X
Project Charter	X
Gates, Reviews and Approvals	X
Benefits-driven Procurement	X
Continuous Risk Management	X
Continuous Risk Mgmt. Guidebook	X
Selected Methods/ Tools/ Techniques	X
Risk Mgmt. Lessons Learned Database	X
Modern Tracking and Control Scenario	X
Function Point Analysis		X
Earned Value		X
Measurement Benchmark Database		X

 

Solutions (Process Solution Set)	Plateau 0	Plateau 1	Plateau 2	Plateau 3	Plateau 4
Capability Maturity Models – SEI		X
Key Process Area Handbooks – SEI		X
EssentialSet Process Templates – SPC		X
IDEAL Guide to Process Improvement	X
Tailoring Guidelines		X
Process Improvement Lessons Learned Database		X
Departmental Assessment Questionnaires	X
CMM-based Appraisal for Internal Process Improvement – SEI		X
Other Capability Assessment Model	X

 

Solutions (People Solution Set)	Plateau 0	Plateau 1	Plateau 2	Plateau 3	Plateau 4
Core Competencies	X
Symposia Series		X
Project Management Guide	X
Project Management Tools Survey	X
Web Site		X
Accreditation Program		X
Project Manager Network		X
Education Network		X

4.3 Project Solution Set

4.3.1 Governance structure

Purpose: This document outlines the governance structure that is to be used across the federal government to manage IT projects. It satisfies the key factors for success as identified in the Chaos Report⁽⁶⁾ – early agreement on project requirements, commitment and involvement of the user community, and attention and decisions by top executives. It is also supported by the concepts of modern comptrollership.⁽⁷⁾

Benefits: A consistent and co-ordinated governance structure is needed for successful implementation of IT projects. A sound governance structure will help departments avoid typical project failures.

Synopsis: The governance structure consists of three elements – a business case, a project charter and the gating, reviews and approval process.

The Business Case puts the investment decision in a strategic context and positions the business objectives and options that will affect both the decision and the investment itself. It provides the information necessary to make a decision about whether a project should proceed. It provides an analysis of all the costs, benefits and risks associated with a proposed investment and with the reasonable alternatives to the proposed investment.

The Project Charter is a signed agreement between all stakeholders that defines the objectives, roles, responsibilities and level of participation of each stakeholder.

The Gating, Reviews and Approval Process establishes in the project charter the project review gates. The review gates are the major decision points of a project – to continue or to walk away. For each gate, the deliverables for that review, the type of review, the stakeholders responsible for reviewing each deliverable and the appropriate approval authorities must be defined.

Affected Audience:

• Business Program executives and Information Technology executives responsible for strategies, priorities, and decisions.
• Business Program managers who implement information technology solutions in their program delivery value chains.
• Information Technology process support teams who provide the common standards and infrastructure for projects.
• Information Technology project teams who need an action ready-project infrastructure so that they can concentrate on delivery of the business solution.

4.3.2 Procurement reform

Purpose: Benefits-driven Procurement (BDP) is a direct response to the stated need for new methods of supply and process that address efficiency and effectiveness concerns. BDP basically alters the way in which requirements are defined, the way in which vendors are solicited and evaluated and the way in which project expectations are managed to ensure success. BDP embraces the principles, practices and tools called for in the Enhanced Framework, and is defined to be a "value-added approach to acquire and implement business solutions for complex IT/SI projects traditionally characterized by significant risk".

The purpose of the Benefits-driven Procurement (BDP) Tool Kit is to provide methods and tools to support the BDP Process. BDP is one of several accepted approaches to procurement.

Benefits: The benefits of Benefits-driven Procurement leading to increased success of the IT/SI projects and, consequently, value for money are:

• flexible business arrangements and processes that are driven by the outcomes and benefits of the project;
• sound, continuous risk management;
• tight integration of the procurement activities with the project management process;
• increased visibility of the fiscal element throughout the project; and
• constant focus on process and organizational ability.

Synopsis: Key elements of the BDP tool kit include:

• the Logical Framework Method as a tool to scope projects;
• a guide to evaluating business cases;
• a guide to evaluating business case tools;
• a risk assessment guide (template);
• a guide to evaluating performance measurement and tracking systems and data;
• a guide to developing a BDP Request for Proposal document (including a template);
• a guide to evaluate BDP proposals (template of evaluation criteria);
• a guide to implementing gating in BDP projects (gating checklist); and
• a guide to contracting under BDP.

Affected Audience: The tool kit has been developed for all stakeholders who will implement, manage or participate in a BDP procurement. It addresses more specifically the needs of the procurement officers who must apply the concepts set forth in the BDP process.

4.3.3 Risk management

Purpose: This solution set is proposed to facilitate the implementation of risk management within IT projects.

Benefits: Sound risk management regimes help people avoid disasters, rework and stimulate win-win situations on software projects. Some of the benefits resulting from implementing risk management include:

• preventing problems before they occur;
• improving product quality;
• enabling better use of resources; and
• promoting teamwork.

Synopsis: The strategy for implementing an improved risk management regime builds upon a series of proven and related elements. These include a Continuous Risk Management approach promoted by the Software Engineering Institute, a Guidebook that identifies and facilitates the use of risk management tools and techniques, a set of selected methods/tools/techniques, and a Lessons Learned database.

Continuous Risk Management is an approach to risk management promoted by the Software Engineering Institute and selected for use by the PMO. Continuous Risk Management is simply an area of emphasis of good project management. It is applied common sense; it should be a normal aspect of the project manager's daily work.

The functions performed in Continuous Risk Management are described in Table 1.

Table 1: Risk Management Functions

Function	Description
Identify	Search for and locate risks before they become problems.
Analyze	Transform risk data into decision-making information. Evaluate impact, probability and timeframe, classify risks, and prioritize risks.
Plan	Translate risk information into decisions and mitigation actions (both present and future), and implement those actions.
Track	Monitor risk indicators and mitigation actions.
Control	Correct for deviations from the risk mitigation plans.
Communicate	Provide information and feedback, internal and external to the project, on the risk activities, current risks, and emerging risks. Note: Communication happens throughout all the activities of risk management.

The Guidebook, published by and available from the Software Engineering Institute explains Continuous Risk Management. It helps practitioners to understand the principles, functions, methods and tools of Continuous Risk Management, shows what it could look like when implemented in a project and how a project can implement its own adaptation. This guidebook is comprehensive and one of the best available on the market. It is an invaluable tool to any department implementing risk management. It does not, however, provide a cookie-cutter solution for all situations. There is no such solution. The guidebook outlines a generic practice with a variety of commonly used methods and tools from which to choose. It is meant to be adapted to suit organizations and projects.

Selected methods/tools/techniques. To help the responsible officials to get started, the following have been selected to initiate improvements. All of these selected approaches are described in the Continuous Risk Management Guidebook, as are several others that may be preferred. Some departments have already used those identified below and therefore provide an opportunity to leverage lessons learned.

Table 2: Selected methods/tools/techniques

Activity	Method/tool/technique
Risk Identification	Taxonomy-based Questionnaire and Interviews Risk Information Sheets
Risk Analysis	Tri-level Attribute Evaluation Taxonomy Classification Comparative Risk Ranking and Top N
Risk Planning	Planning Decision Flow Chart Risk Information Sheets
Risk Monitoring	Tri-level Attribute Evaluation Risk Information Sheets
Risk Control	Cause and Effect Analysis Risk Information Sheets

Lessons Learned database: The PMO is in the process of implementing a risk management lessons learned database that is scheduled to be operational in the fall of 1998. The purpose of this database is to document what government departments have read, learned, tested and experienced in risk management as well as documenting successes and less-than successful experiments.

Affected Audience: Improved risk management will have implications for two distinct groups within the federal government IT community:

• the departments that must manage and deliver IT projects (specifically the Head of IT to implement the program and the Project Managers to use the tools); and
• the agents, such as PWGSC, who must ensure that the acquisition vehicles used for IT goods and services embody the principles of improved risk management and see to it that these are respected by private-sector suppliers.

4.3.4 Planning and control

Purpose: The purpose of Project Tracking and Control Standards is to ensure that projects are planned consistently and the right information is available to support both routine and milestone decisions. A common set of expectations and vocabulary will help users to exploit the power of information technology.

Benefits: Co-ordinated and consistent Project Tracking and Control Standards are needed for successful implementation of information technology projects in industry and government. Key factors for success include early agreement on project requirements; commitment and involvement of the user community; and attention and decisions by top executives. A core set of definitions and measures will streamline business communications and business decisions.

Synopsis:
Summarized components of a modern Tracking and Control scenario include:

• roles and responsibilities of project sponsors, executive managers, project leaders, project managers, support teams, project teams, operations teams;
• decision loops, decision points and decision processes at both the organizational and project levels;
• reporting standards for each decision point within the decision loop; and
• a Measurement Benchmark Database, which will be established to support the realistic evaluation of size, cost, schedule and risk.

The Earned Value Method of planning and reporting progress on deliverables, which is being used for Major Crown Projects, will be adapted for use in information technology projects.

The Function Point Method of planning and controlling the size of software will be packaged for implementation.

Affected Audience:

• Business Program executives and Information Technology executives responsible for strategies, priorities and decisions.
• Business Program managers who implement information technology solutions in their program delivery value chains.
• Information Technology process support teams who provide the common standards and infrastructure for projects.
• Information Technology project teams who need an action-ready project infrastructure so that they can concentrate on delivery of the business solution.

4.4 Process Solution Set

4.4.1 Process improvement models and guides

Purpose: This solution set is intended to ease the implementation of process improvements within IT organizations across the federal government in support of the Enhanced Framework.

Benefits: Some of the recorded and publicly announced benefits resulting from process improvement⁽⁸⁾ include:

• increases in productivity ranging from 10 per cent to 100 per cent;
• defect and error rate reductions of 45 per cent to 70 per cent;
• cost savings ranging from $4 to $6 saved for every dollar invested in process improvement;
• schedule improvements, including the reduction from 50 per cent to 1 per cent late projects; and
• rework reduced from 40 per cent to 25 per cent of the total project effort.

It is reasonable to assume that similar gains can be achieved in the federal government and that these improvements will provide a level of predictability in IT projects that will minimize project cancellations and/or overruns.

Synopsis: The Process Improvement implementation strategy builds upon a series of proven and related elements. These include: Capability Maturity Models, which identify key practices that must be performed in the project environment; guides and templates that describe these practices; tailoring guidelines and support; an improvement path that clearly identifies the milestones to be achieved; an improvement process to guide departmental initiatives and a lessons learned database.

The Capability Maturity Models (CMMs) were created and are promoted by the Software Engineering Institute (SEI). While the initial focus is on the software CMM since software is an area of prime PMO concern, other models such as the Systems Engineering CMM, the Software Acquisition CMM and the People CMM are also available. They can be accessed on the Internet or through the PMO for those situations in which they should be applied. Each CMM defines a number of Key Process Areas, and is comprised of Goals and Key Practices that must be followed to assure an organizational capability to deliver on project commitments.

Personal Software Process (PSP): The original impetus for developing the PSP came from questions about the SEI software-CMM. Many viewed the CMM as designed for large organizations and did not see how it could be applied to individual work or to small project teams. As a result, the SEI has devised a means of adapting 12 of the 18 CMM key process areas to the work of individual software engineers and this is expressed in the PSP.

It has been found that, by using a defined and measured personal software process, engineers can improve the quality of their products by five to ten times while also improving their productivity. The PSP is a promising way for engineers to understand their own performance and to see how to improve it.

Key Process Area Handbooks fully describing each of the 18 SW-CMM Key Process Areas (e.g. Requirements Management) have been assembled from the SEI materials. Information includes an overview, a description of the goals, practices and activities, roles, inputs, outputs, entry and exit criteria, measurements, work products managed and controlled, documented procedures, training and tools. These are available from the PMO.

EssentialSet is a set of adaptable and reusable templates based on the SEI SW-CMM that define how to perform the SEI Key Practices in order to fully satisfy the goals of Enhanced Framework Plateaux 1 and 2. It is available through the Software Productivity Centre in Vancouver (http://www.spc.ca).

Tailoring is an important component or tool in the solution set. It is expected that the SEI models, Key Process Areas and Key Practices, and tools built on them such as EssentialSet, will be tailored by the organization before being applied. A guide for tailoring the CMM for small - to medium-sized organizations is available from LOGOS, Inc. in the U.S. by calling (617) 259-8266.

The Improvement Path is described more fully in Section 3 of this document. It is designed to be directly applicable to the Process Improvement initiative.

The IDEAL User's Guide for Process Improvement (also an SEI product and available through both the SEI and the PMO) can be used as a step-by-step guide to implement improvements in an organization. It provides a comprehensive, step-by-step guide on how to introduce change in a structured manner. It saves time by:

• defining clear roles and responsibilities;
• ensuring that no steps are missed; and
• leveraging lessons learned through a cyclical approach.

A Process Improvement Lessons Learned database is being developed by the PMO.

Affected Audience: This solution is intended primarily for the providers of IT goods and services. They can be represented by departmental IT managers and practitioners, as well as the IT industry, which needs to improve its processes in order to satisfy the ever-increasing pressures placed on them.

4.4.2 Assessment methodologies and tools

Purpose: The Assessment and Methodologies Tools outlined below can be used by IT organizations to determine the areas where best to assign resources for greatest return on investment in support of the Enhanced Framework.

Benefits: Benefits resulting from assessments include:

• identification of strengths and areas for improvement;
• guidance in selecting and rolling out EMF Solutions;
• identification of opportunities to leverage best practices;
• establishment of a baseline against which to measure progress; and a
• kick-start for new improvement initiatives or support for ongoing activities.

Synopsis: The PMO and departments have used a number of tools to assess compliance with the Enhanced Management Framework and an organization's capability maturity.

The Enhanced Framework Questionnaire was designed to assess the degree to which departments were using recommended practices. It is a requirement that must be completed by departments seeking TBS approval for a project.

Each SEI CMM has a standard and comprehensive assessment tool (questionnaire) and methodology associated with it that can be used to determine the degree to which an organization is capable of consistently performing well-defined Key Processes. These tools point to specific practices requiring improvement.

The most comprehensive of these is the CMM-based Appraisal for Internal Process Improvement, or CBA-IPI. An SEI-trained and authorized lead assessor must conduct it. It can take 200 person-days over several months and cost about $100K, but it has been proven to work in many organizations. SEI has also developed a Maturity Questionnaire that serves primarily to identify issues requiring further exploration. It addresses each Key Process Area goal, but not all of the Key Practices. There are six to eight questions for each of the 18 Key Process Areas.

S:PRIME is a methodology and a product based on the SEI SW-CMM. It can be used to identify and prioritize improvements to software processes based on risk mitigation and to develop an action plan. It is available through the Applied Software Engineering Centre (ASEC) in Montreal by calling (514) 840-1244.

Softguide is an automated questionnaire used to identify and prioritize areas for improvement and develop an action plan. It is available through the Software Productivity Centre (SPC) in Vancouver (http://www.spc.ca)

The Systems Engineering CMM has a questionnaire associated with it as does the Software Acquisition CMM. Both are available through the SEI.

The PMO has put together an Assessment Questionnaire comprised of 117 questions that will be used to assess process capability in departments. Twenty of the questions are from the original EMF Project Assessment Questionnaire, eight are from the SE-CMM questionnaire and 89 are modified Softguide questions.

Another related solution of interest is SEI's Software Capability Evaluation (SCE) Methodology. The SCE is used for software acquisition as a discriminator to select suppliers, for contract monitoring and for incentives. It can also be used for evaluation of internal processes. The SCE is used to gain insight into the software process capability of a supplier organization and is intended to help decision makers make better acquisition decisions, improve subcontractor performance and provide insight to a purchasing organization.

Affected Audience: This solution is intended primarily for the providers of IT goods and services. They can be represented by departmental IT managers and practitioners, as well as the IT industry who need to improve their processes in order to satisfy the ever-increasing pressures placed on them.

4.5 People Solution Set

4.5.1 Project management support

Purpose: The PM Support program is designed to provide project sponsors, project leaders and project managers with the appropriate procedures, tools and training to be able to successfully manage IT projects within the federal government environment.

Synopsis: The project manager support program has four components: the Project Management Symposia series; a Project Management Guide; a survey of project management tools; and the Enhanced Framework web-site.

The Core Competencies document details the core competencies, or basic skills, required by persons managing an information technology (IT) project in the Canadian federal government be they the project sponsor, project leader or project manager. These skills fall into three broad areas: general management, project management and IT/IM project management.

The PM Symposia Series consists of three symposia all focusing on managing IT projects within the federal government environment. The first symposium will be a two-day event for executive-level project sponsors and project leaders. The second is a 10-day symposium for master project managers. (A five-day subset of this symposium will be offered to intern/professional project managers). The third is a five-day symposium for non-executive-level project sponsors and project leaders.

The Project Management Guide is to provide a simple but comprehensive guide to the management of IT projects in the federal government. It is a hierarchy of documents that covers all aspects of project management related to the governance of the project lifecycle. The contents of the guide are based on the ISO/IEC 12207 software lifecycle process descriptions and will incorporate principles of the Project Management Institute's Project Management Body of Knowledge (PMBOK), the Software Engineering Institute's Capability Maturity Model for Software and other relevant TBS and external reference materials. In addition to the project manager's role, the document will also address the roles of the project leader and project sponsor.

The PM Tools Survey identifies and describes a set of effective automated tools to support project management activities. It defines the tools and explains how the features of current products can improve system development and maintenance.

The Enhanced Framework Web-site, will contain the documents and lessons learned databases developed for the project management solution set. It will also contain hyperlinks to key project management related web-sites.

Affected Audience: Project sponsors, project leaders and project managers would benefit from the symposia series. The guide and tools survey are intended primarily for the use of departmental process improvement personnel to determine where and how the solutions of the Enhanced Management Framework should be incorporated into the departmental project life cycle. These tools are also intended to provide support to IT project personnel as they implement the principles of the Enhanced Framework within their projects.

4.5.2 Project management career development

Purpose: The purpose of the PM Career Development program is to ensure that project managers have the required skills and experience to manage large IT projects.

Benefits: With accredited project managers, projects will be better managed and therefore have a higher success rate. The PWGSC Institute will maintain an inventory of all project managers resulting in the creation of a Project Management community. Once the PM community has been identified, career development and enhancement opportunities will be created and programs like a project management network can be developed. In addition, with accreditation, a career development target for project management will exist for staff and management.

Synopsis: The career development program consists of an education program, an accreditation program and a project manager network.

The Education Program, called The Institute Certificate in Project Management, is available from The Institute for Government Information Professionals in PWGSC. It currently consists of seven courses and three seminars.

The Accreditation Program will also be available from The Institute for Government Information Professionals in PWGSC. It will consist of an exam to ensure adequate skills and the requirement to have a minimum of 4 years experience in managing complex IT projects and two recommendations from project sponsors.

The Project Manager Network will be developed once the inventory of project managers has been created. The intent is to establish a network for the exchange of information and lessons learned.

Affected Audience

Aspiring project managers and their management will use the PM career development program. The Treasury Board will require that projects seeking TBS approval have an accredited project manager identified for the project.

4.6 Solution Availability

Some of the solutions described have already been developed and are available from the TBS, while others are still in the development stages. The availability of the solutions is summarized in the following chart:

Solution	December 1997	March 1998	June 1998
Business Case	x
Project Charter		x
Gating, Reviews and Approval Process		x
Procurement Reform	x
Risk Management	x
Planning and Control		x
Measurement Benchmark Database			x
Process Improvement Models and Guides	x
Assessment Methodologies and Tools	x
Project Management Support – Symposia Series			x
Project Management Guide		x
PM Tools Survey		x
Enhanced Framework Web-site		x
Project Management Career Development Program			x
Lessons Learned		x

A complete list of all developed supporting documentation available from the TBS or from its original source can be found in Appendix 3.

5. How to Get Started

The IDEAL^SM Model⁽⁹⁾ has been selected as a guide for departmental improvement efforts. It is described more fully in Appendix 4. The IDEALSM Model below depicts the five phases of process improvement that provide a continuous loop through the steps necessary for improvement.

A 'big bang' approach that could dictate a long wait for improvements is not recommended. Rather, it is beneficial to address small improvements through the completion of as many cycles as necessary to achieve the desired Plateau depicted in Figure 2. While the length of time it should take to complete a cycle is not specified, a target of 6-12 months is recommended.

Figure 5: IDEALSM Model

6. Conclusion

This document has addressed the implementation of the concepts and precepts defined in An Enhanced Framework for the Management of Information Technology Projects (Enhanced Framework) approved and published in May 1996. TBS has directed departments to apply this framework to existing projects, as applicable, and to all future information technology projects.

Implementation of the Enhanced Framework is designed to ensure that government information technology projects fully meet the needs of the business functions they are intended to support, deliver all expected benefits, and are completed on time, to cost and functionality. It is also anticipated that implementation will, over time, enable departments to increase their levels of authority.

Canada is establishing itself as a leader in the field of IT/IM project management and delivery. To the best of the Treasury Board's knowledge, no government in the world has put in place such an ambitious program. With IT/IM being so crucial to the efficient and effective delivery of its programs and services, Canada cannot afford not to harvest the benefits of the technology it deploys every year.

Implementation of the Enhanced Framework constitutes a significant step towards operational effectiveness within the government. The opportunity offered to senior government managers to embrace a proven solution to a widely recognized problem should not be missed.

APPENDIX 1

ASSESSMENT QUESTIONNAIRE

This Appendix contains the Assessment Questionnaire. This Assessment Questionnaire is comprised of 117 questions that can be used assess current practices in departments. Twenty of the questions are from the original Enhanced Framework Assessment Questionnaire, eight are from the SEI-CMM questionnaire and 89 are modified Softguide questions.

Appendix 1a
Assessment Questionnaire against the Enhanced Framework
for the Management of IT Projects

Framework Element	Yes	No	Don't Know	N/A	Comments
1. Principle: Information technology projects are aligned with, and support, business directions and priorities
Is approval for the project based on a business-case analysis that relates the investment directly to the business function and demonstrates the benefits of the investment to the department or government as a whole?
Is there a business case template used by all projects?
Is there a documented process for preparing the business case, obtaining approvals, reviewing the business case, etc?
Will the business case be reviewed and revalidated at each scheduled gate and whenever there is a significant change to the project or the business function?
2. Principle: Clear accountabilities are established
Are overall departmental accountabilities for the project defined in a project charter?
Is there a project charter template used by all projects?
Is there a documented process for preparing the project charter, identifying roles and responsibilities of stakeholders, obtaining approvals, etc?
Are all core project management responsibilities and functions performed by Crown management? (This includes core functions associated with managing and controlling project plan, scope, time, cost, quality, risk, human resources, procurement, contract and communications.)
If it has been necessary to outsource any core project management functions, are they being acquired from a supplier other than that involved in the primary development contract?
3. Principle: Project managers are developed and work within a corporate discipline
Does the assigned project manager have the knowledge, skills and experience required to manage the project's scope, size, complexity and risk profile?
Are the competencies expected and required of project managers documented?
Is there a documented process that defines a training and development path for project managers?
4. Principle: Project management decisions are based on risk management
Does the project have scheduled checkpoints or 'gates' when it will be reviewed and where management will decide on its future and, if necessary, take appropriate corrective action?
Does the department's defined project life cycle detail when and where gates should be established and what criteria must be satisfied before clearing the gate?
Have only the funds needed to reach the next gate been allocated to the project?
Has a risk assessment consistent with practices outlined in SEI Continuous Risk Management Guidebook been used to identify and assess the risks?
Is there a documented risk management process (including tools, techniques and practices) that all projects must use?
Has project complexity been determined at the initiation of the project using Function Point Analysis (FPA)?
Is a performance measurement tool based on the national standard, C/SPMS, being used to provide data at frequent intervals to the (Crown) project manager on the time and money expended and on the work completed?
Have PWGSC procurement officers been involved early in the project planning so as to develop a procurement process that reduces delays, and to design a procurement plan that best aligns the contracting plan with the project plan?

Appendix 1b
Departmental Assessment – System Engineering Questions

System Engineering Questions	Yes	No	Don't Know	N/A	Comment/ Explanation
1. Is there a documented process to elicit, stimulate, analyze, and communicate customer needs and expectations to obtain a better understanding of what will satisfy the customer? (This involves engaging the customer in ongoing dialogue designed to translate his/her needs and expectations into a verifiable set of requirements that the customer understands and that provide the basis for agreements between the customer and the project.)
2. Are candidate solutions studied and/or analyzed prior to selecting a solution to satisfy the business problem and its constraints?
3. Are business requirements allocated/mapped to various system functions (including objects, people, supporting processes, products and services) as part of the system design activities and/or the selection of off-the-shelf solutions?
4. Are technical architecture requirements derived and allocated as part of the system design activities?
5. Is there a process whereby software acquisitions are planned, including establishing budgets, determining the schedule, defining an acquisition strategy, identifying risks, and defining requirements for the software product?
Do documented guidelines and procedures exist for the acquisition of software products?
For the software product being acquired, is transition to the software support organization planned?
Is there a documented process to ensure that system elements will function as a whole? (This primarily involves identifying, defining and controlling interfaces, as well as verifying system functions that require multiple system elements.)

Appendix 1c
Assessment Questionnaire – Using Capability Maturity Model Questions

Framework Element	Yes	No	Don't Know	N/A	Comment/ Explanation
1. Requirements Management
1. Are requirements agreed upon by the customer, project management and project team members?
2. Are requirements documented and available to all project team members?
3. Are changes to requirements reflected in the project's effort and cost estimates, and size of product estimates?
4. Are changes to requirements negotiated and agreed upon by the customer, project management and project team members?
5. Are requirements traced to design components, code components and test plans or procedures?
6. Are requirements analyzed for completeness, understandability, feasibility and consistency?
2. Project Planning
1. Are the project activities defined and documented in a plan? (Project Plan)
2. Do procedures or guidelines exist for estimating project effort and cost, and size of work products?
3. Are the commitments of external groups documented and agreed upon by the affected groups (i.e. configuration management, documentation management, quality assurance, customer and sub-contractors)?
4. Is the project plan reviewed by the project manager, managers of other affected groups, and project team members?
5. Are project risks (cost, resource, schedule, or technical) identified, assessed and documented?
6. Are the planned activities of the project based upon a defined life cycle?
7. Have facilities (i.e. office space, computer equipment) and support tools been identified and procured or developed?
3. Project Tracking and Oversight
1. Are the project effort, cost, and schedule tracked on a frequent basis?
2. Are sizes of work products tracked and the sizing estimates updated accordingly?
3. Are the project activities adjusted and re-planned when the project actuals are found to deviate significantly from the original project plan?
4. Are internal project reviews conducted periodically with affected groups to track progress and issues?
5. Are reviews conducted at significant milestones with the customer to review results, plans and status?
6. Are estimated and actual data for project effort, cost, and work product sizes recorded for use in this project and future projects?
7. Are technical issues or problems identified, documented, and tracked to closure (i.e. problem reports, issues database)?
4. Subcontract Management
1. Are subcontractors selected based upon an evaluation of their capabilities?
2. Do documented guidelines or procedures exist for the evaluation of subcontractors?
3. Are the commitments between the prime contractor and the subcontractor documented in a contractual agreement?
4. Does the prime contractor conduct periodic technical reviews with the subcontractor to review technical material and status?
5. Does the prime contractor conduct periodic management reviews with the subcontractor to review progress and status?
6. Does the prime contractor conduct an acceptance test as part of the criteria for accepting the subcontractor's product?
7. Is the subcontractor's performance evaluated periodically and reviewed with the subcontractor?
8. Are the product acceptance criteria documented in an Acceptance Test Plan, which has been agreed upon by the prime contractor and the subcontractor?
5. Quality Assurance
1. Does a Quality Assurance (QA) Plan, containing QA activities, responsibilities, and schedule, exist for the project?
2. Are QA activities conducted in accordance with the QA Plan?
3. Does Quality Assurance report directly to the organization's senior management?
4. Are QA activities and their findings reported to the project periodically?
5. Are QA individuals trained in quality assurance?
6. Are issues of non-compliance that cannot be resolved within the project, escalated to the level of senior management?
7. Does QA verify that the activities of the project comply with the Project Plan and the designated standards and procedures identified in the Project Plan?
8. Does QA verify that work products comply with the standards, procedures, and contractual requirements, as stipulated or referenced by the Project Plan and Statement of Work?
6. Configuration Management
1. Does a configuration management (CM) Plan, outlining CM activities, responsibilities, and schedule, exist for the project?
2. Are CM activities conducted in accordance with the CM Plan?
3. Are project work products, supporting tools, and any software or procedures required to regenerate the work products identified and controlled?
4. Does CM library system serve as a repository for the controlled items?
5. Is there an established procedure for checking items in and out of the CM library system?
6. Is there an established procedure for generating baselines from the CM library system?
7. Is information on the contents of the baselines and the status of the CM library available to the project?
8. Is there an established procedure and mechanism for controlling changes to the controlled items (i.e. Change Request Procedure)?
7. Organization Process Focus
1. Do activities to develop standardized processes and to improve them exist within the organization?
2. Does responsibility for defining process development and improvement exist at an organizational level, and not a project level?
3. Are the project's or organization's processes assessed periodically to determine process strengths and weaknesses?
4. Are action plans based upon the process assessment generated and implemented?
5. Does the organization have a company-wide plan for process development and improvement activities?
6. Is there an Organizational Training Plan for conducting process training across the organization?
7. Does senior management provide the support, resources, and funding to enable the process improvement activities to be effective?
8. Organization Process Definition
1. Does the organization have developed and documented standardized processes (i.e. description of project life cycle)?
2. Does the organization encourage and promote the use of standardized processes?
3. Is there a collection of process assets (e.g. process descriptions, tailoring guidelines, coding standards, development procedures) that can be easily tailored and used by the projects?
4. Are the standard processes continually assessed and improved?
5. Are new technology, tools, and methodologies related to process being assessed and evaluated?
6. Are estimated and measured process data retained in a database for use in process improvement and in planning future projects (e.g. size estimates, effort data, productivity data, defect data)?
9. Training Program
1. Does each project specify its technical and managerial training needs (e.g. type of training required, by whom, and when)?
2. Do the individuals on a project receive the necessary training as identified?
3. Does a training plan for the organization exist, specifying training needs, type of training available, funding, resources, schedules, and standards for course development?
4. Are training courses produced in-house, developed and maintained according to the organization's training plan?
5. Does the organization's training program receive the necessary support, resources, and funding from senior management to make the program effective?
6. Are measurements used to determine the quality of the training program?
10. Integrated Project Management
1. Does the project have defined processes that have been developed from the organization's standard processes?
2. Do the activities described in the Project Plan follow the project's defined processes?
3. Are historical data from past projects (as contained in the organization's project process database) used for project planning and estimating? (These historical data include software size, effort, cost, schedule, productivity and activity data.)
4. Are the project's effort, costs and schedule managed according to a procedure?
5. Is the size of the work products managed according to a procedure?
6. Are the project's risks identified, assessed, documented and managed according to a procedure?
11. Software Product Engineering
1. Is the project's defined software process supported by effective mechanisms, and tools?
2. Are procedures for using the defined software process documented and adhered to?
3. Are requirements analysis and verification conducted in accordance with the project's defined process?
4. Is the product designed in accordance with the project's defined process?
5. Is the product implemented according to the project's defined process?
6. Is the testing conducted according to the project's defined process?
7. Are the work products (i.e. outputs) of the requirements analysis, design, implementation, and test activities consistent with each other and the customer requirements?
8. Are the necessary resources (i.e. technology, skills, equipment) available to the project?
12. Inter-group Co-ordination
1. Are commitments between project groups agreed to by the affected groups and documented? (Commitments could be documented in the Project Plan.)
2. Are representatives from affected project groups involved in establishing the project requirements and in negotiating with the customer?
3. Does an established procedure exist for identifying, recording, tracking and closing inter-group issues?
4. Do the project groups work together on a regular basis to monitor and co-ordinate technical activities and to resolve technical issues?
5. Is the Project Plan used to co-ordinate and track the activities of the various groups?
6. Are work products delivered to other project groups reviewed to ensure they meet the group's needs?
13. Peer Reviews
1. Is peer review of work products conducted on the project? Examples of work products suitable for review are requirement specifications, architecture, design descriptions and test plans.
2. Is the review material distributed to the reviewers prior to the meeting and with sufficient time allocated for the material to be reviewed?
3. Are defects identified during the review meeting recorded and tracked to closure?
4. Does the Project Plan or a Peer Review Plan identify the work products to undergo peer review?
5. Are peer reviews conducted in accordance with a documented procedure?
6. Does the reviewer use peer review checklists, which identify the criteria for the review of the product?

APPENDIX 2

SPECIFIC GOALS TO BE MET AT PLATEAUX 0 – 4

This Appendix outlines the goals to be achieved at each Plateau of the Departmental Improvement Plateaux as part of Enhanced Framework implementation. Plateau 0 and Plateau 1 goals are primarily project-oriented while Plateau 2 goals are both project and product oriented. Organization-oriented goals reside in the third Plateau and Plateau 4 goals are oriented towards the achievement of continuous improvement.

Plateau 0 Goals: Baseline

Basic Project Management

Goal 1

Implement the basic solutions required by the Enhanced Framework and described in this document. For each project these include a clear and explicit business case, a procurement strategy, a project charter, a gating and review process, a project planning and control mechanism and a risk management approach.

Improvement Planning

Goal 2

Define a plan for advancing to the next Plateau and a means for measuring progress.

Plateau 1 Goals: Project Focus

Project Planning

Goal 1	Estimates are documented for use in planning and tracking the project.
Goal 2	Project activities and commitments are planned and documented.
Goal 3	Affected groups and individuals agree to their commitments related to the project.

Project Tracking and Oversight

Goal 1	Actual results and performances are tracked against the plans.
Goal 2	Corrective actions are taken and managed to closure when actual results and performance deviate significantly from the plans.
Goal 3	Affected groups and individuals agree to changes to commitments.

Subcontract Management

Goal 1	The prime contractor selects qualified subcontractors.
Goal 2	The prime contractor and the subcontractor agree to their commitments to each other.
Goal 3	The prime contractor and the subcontractor maintain ongoing communications.
Goal 4	The prime contractor tracks the subcontractor's actual results and performance against its commitments.

Plateau 2 Goals: Product Focus

Requirements Management

Goal 1	System requirements are controlled to establish a baseline for software engineering and management use.
Goal 2	Plans, products and activities are kept consistent with the system requirements.

Configuration Management

Goal 1	Configuration management activities are planned.
Goal 2	Selected work products are identified, controlled and available.
Goal 3	Changes to identified work products are controlled.
Goal 4	Affected groups and individuals are informed of the status and content of baselines.

Quality Assurance

Goal 1	Quality assurance activities are planned.
Goal 2	Adherence of products and activities to the applicable standards, procedures and requirements is verified objectively.
Goal 3	Affected groups and individuals are informed of quality assurance activities and results.
Goal 4	Senior management address non-compliance issues that cannot be resolved within the project.

Plateau 3 Goals: Process Focus

Organizational Process Focus

Goal 1	Software process development and improvement activities are co-ordinated across the organization.
Goal 2	The strengths and weaknesses of the software processes used are identified relative to a process standard.
Goal 3	Organization-level process development and improvement activities are planned.

Organization Process Definition

Goal 1	A standard software process for the organization is developed and maintained.
Goal 2	Information related to the use of the organization's standard software process by the software projects is collected, reviewed and made available.

Training Program

Goal 1	Training activities are planned.
Goal 2	Training for the development of the skills and knowledge needed to perform software management and technical roles is provided.
Goal 3	Individuals in the software engineering group and software-related groups receive the training necessary to perform their roles.

Integrated Project Management

Goal 1	Every project's defined software process is a tailored version of the organization's standard software process.
Goal 2	Every project is planned and managed according to the project's defined software process.

Software Product Engineering

Goal 1	Software engineering tasks are defined, integrated and consistently performed to produce the software.
Goal 2	Software work products are kept consistent with each other.

Inter-group Co-ordination

Goal 1	All affected groups agree to the customer's requirements.
Goal 2	Commitments between the engineering groups are agreed to by the affected groups.
Goal 3	The engineering groups identify, track and resolve inter-group issues.

Peer Reviews

Goal 1	Peer review activities are planned.
Goal 2	Defects in the software work products are identified and removed.

Plateau 4: Continuous Improvement

Goal 1	Quantitative process management activities are planned.
Goal 2	Process performance of the project's defined software process is controlled quantitatively.
Goal 3	Process capability of the organization's standard software process is known in quantitative terms.
Goal 4	The project's software quality management activities are planned.
Goal 5	Measurable goals for software product quality and their priorities are defined.
Goal 6	Actual progress towards achieving the quality goals for the software products is quantitatively managed.
Goal 7	Defect prevention activities are planned.
Goal 8	Common causes of defects are sought out and identified.
Goal 9	Common causes of defects are prioritized and systematically eliminated.
Goal 10	Incorporation of technology changes is planned.
Goal 11	New technologies are evaluated to determine their effect on quality and productivity.
Goal 12	Appropriate new technologies are transferred into normal practice across the organization.
Goal 13	Continuous process improvement is planned.
Goal 14	Participation in the organization's software process improvement activities is organization-wide.
Goal 15	The organization's standard software process and the projects' defined software processes are improved continuously.

APPENDIX 3

LIST OF MATERIALS AVAILABLE FROM THE CIOB

The materials listed in this Appendix are available from the CIOB at the Treasury Board of Canada Secretariat.

For general enquiries, please send e-mail to emf-cag@tbs.sct.gc.ca 

Title	Description	General Inquiries
Governance Structure
Creating and Using a Business Case for IT Projects	A guide for IT and Responsibility Centre Managers	emf-cag@tbs-sct.gc.ca
Procurement Strategy
Benefits-driven Procurement (BDP) Concept Paper	Paper describing concept and philosophy of BDP	www.cata.ca/cata/advocacy/
BDP Process Document	Paper outlining BDP process
Risk Management
Enhanced Framework Questionnaire	Assesses a project's compliance with the Enhanced Framework – required to support TBS submissions	emf-cag@tbs-sct.gc.ca
Guides to Successful IT Projects	Questionnaires designed as guides for senior managers to help them 'ask the right questions'	emf-cag@tbs-sct.gc.ca
Continuous Risk Management (CRM) Guidebook	Explains Risk Management and describes the associated principles, functions, methods and tools	www.sei.cmu.edu/ *
Software Acquisition Risk Mgmt Key Process Area Guidebook	Describes the Risk Management key process area for the Software Acquisition CMM	emf-cag@tbs-sct.gc.ca
Introduction to Team-based Risk Management	Describes processes, methods and tools to support joint risk management by the government and the supplier	www.sei.cmu.edu/ *
Taxonomy-based Risk Identification	Method for facilitating systematic and repeatable risk identification in software projects	www.sei.cmu.edu/ *
S:PRIME	SEI-based methodology and tool supporting risk mitigation via process improvement	GRafP (514) 840-1244
Lessons Learned database	Captures lessons learned from projects and pilots	Under construction
Planning and Control
A Planning and Control Guideline	Describes the process that middle managers (EX-1/2) should use to control progress in maintenance, enhancement and development	Under construction
Process Improvement Models and Guides
Capability Maturity Model for Software V1.1	Overview of SW-CMM describing the maturity framework's 5 levels and 18 key practice areas (KPAs)	www.sei.cmu.edu/ *
Key Practice Areas of the CMM	Describes the over 300 key practice areas for the SW-CMM	www.sei.cmu.edu/ *
Key Practice Area Handbooks	All SEI materials on the 18 KPAs (e.g. everything about project planning and tracking)	www.sei.cmu.edu/ *
EssentialSet Templates	54 reusable and adaptable templates to help manage and control IT projects	Software Productivity Centre www.spc.ca
Software Process Framework	Guide on how to design or verify practice consistency with SW-CMM	www.sei.cmu.edu/ *
Process Tailoring and the CMM	Guide on how to tailor CMM	www.sei.cmu.edu/ *
Tailoring the CMM for small organizations and projects	Guide on how to tailor the CMM	LOGOS International Inc. (617) 259-8266
Software Acquisition CMM SA-CMM	Provides a framework for assessing and improving software acquisition processes	www.sei.cmu.edu/ *
Systems Engineering CMM (SE-CMM)	As above but for software engineering	www.sei.cmu.edu/ *
People CMM	As above for human resources management	www.sei.cmu.edu/ *
Personal Software Process (PSP)	Methodology for significantly improving KPA performance on a personal level	www.sei.cmu.edu/ *
IDEAL Users Guide for Process Improvement	Guide on how to implement a process improvement program	www.sei.cmu.edu/ *
Comparison of SW-CMM and ISO 9001	A detailed comparison and discussion of relationships	www.sei.cmu.edu/ *
Memory Jogger II	Pocket guide of tools for continuous improvement and effective planning	GOAL / Quality Productivity Council 1-800-643-4316
Lessons Learned database	Captures lessons learned from projects, pilots	Under construction
Assessment Methodologies and Tools
CMM-based Appraisal for Internal Process Improvement CBA-IPI	Formal SW-CMM assessment methodology	www.sei.cmu.edu/ *
Maturity questionnaire	Questions about the implementation of important software processes	www.sei.cmu.edu/ *
S:PRIME	CMM-based methodology and tool supporting risk mitigation via process improvement	GRafP Technologies
SoftGuide	CMM-based tool supporting capability assessment and action planning	Software Productivity Centre www.spc.ca
SA-CMM SEC-MM Questionnaires	Questionnaires to assess SE, SA capability against established KPAs	www.sei.cmu.edu/ *
Self Diagnostic Questionnaire	Modified Enhanced Framework, SE-CMM, SoftGuide questions	emf-cag@tbs-sct.gc.ca
Software Capability Evaluation (SCE)	Discriminator used to select suppliers and monitor contracts	www.sei.cmu.edu/ *
Project Management Support
Project Management Core Competencies	Describes the core competencies for project sponsors, leaders and managers	emf-cag@tbs-sct.gc.ca
Symposia Series	Two, 5- and 10-day symposia on managing projects in the federal government environment	Under construction
Project Management Guide	Simple, comprehensive guide on managing projects in the government environment	Under construction
PM Tools Survey	Identifies automated tools to support project management	Under construction
Guide to the Project Manager Body of Knowledge	Describes the sum of knowledge within the profession of project management	Project Management Institute (704) 586-3715
Enhanced Framework Web site	A compendium of project management knowledge and lessons learned	Under construction
Bookmarks	An organized list of project management and software process improvement web sites	emf-cag@tbs-sct.gc.ca
Project Management Career Development
Canadian university and college programs	Programs in Computer Science, Project Management and Management Information Systems	emf-cag@tbs-sct.gc.ca
Education Program	Based on core competencies, leads to Institute Certificate in Project Management	Under construction
Certificate Program	Requires examination plus experience	Under construction
PM Network	An established community of Project Managers	Under construction
Other General Enhanced Framework Publications and Material
Enhanced Framework for the Management of Information Technology Projects	Describes an enhanced framework and action plan to improve government project management	www.tbs-sct.gc.ca/
Enhanced Framework Committees and Working Groups	Membership and mandates	emf-cag@tbs-sct.gc.ca
Enhanced Framework contacts	Departmental and private-sector contacts with experience in implementing elements of the Enhanced Framework	emf-cag@tbs-sct.gc.ca
A Cultural Change Framework	Describes the roles and responsibilities of the various players required to successfully implement change	emf-cag@tbs-sct.gc.ca
A Communications Approach to Implementation	Describes the players, the messages and the media required to communicate and implement change	emf-cag@tbs-sct.gc.ca
International Standard ISO 12207 (TBITS 27)	Describes primary, supporting and organizational Software Life-Cycle Processes	Infrastructure Investment Management Group (613) 957-2496

* Also available from the CIOB in hard copy

APPENDIX 4

IDEAL^SM MODEL: AN OVERVIEW

The following paragraphs provide a general definition of each of the model's phases and are an adaptation from the SEI Handbook IDEAL^SM: A User's Guide for Software Process Improvement (CMU/SEI-96-HB-001).

It should be noted that, to keep the document to a manageable size, details of the SEI IDEAL^SM Model have not been repeated. A summary is provided below, but readers may want to obtain the documentation found on the WWW by SEI at their site http://www.sei.cmu.edu or through the PMO.

Initiating Phase:

The Initiating phase of the IDEAL^SM model is the starting point. Here is where the initial improvement infrastructure is established, the roles and responsibilities for the infrastructure are initially defined, and initial resources are assigned. In this phase, the Enhanced Framework Implementation Plan is created to guide the organization through the completion of the Initiating, Diagnosing and Establishing phases. Approval for the initiative is obtained along with a commitment of future resources for the job ahead. The general goals of the program are defined during the Initiating phase. They are established from the business needs of the organization and will be refined and made specific during the Establishing phase of IDEAL^SM.

Two key components are typically established: a management steering group and a process group. Also during the Initiating phase, plans are made for communicating the start of the initiative, and it is suggested that organizational assessments be performed to determine the readiness of the organization for an improvement initiative.

Diagnosing Phase:

The Diagnosing phase of the IDEAL^SM model starts the organization on the path of implementation. This phase lays the groundwork for the later phases. In this phase, the action plan is initiated in accordance with the organization's vision, strategic business plan, lessons learned from past improvement efforts, key business issues faced by the organization, and long-range goals. Appraisal activities are performed to establish a baseline of the organization's current state. The results and recommendations from appraisals and any other baseline activities will be reconciled with existing and/or planned improvement efforts for inclusion in the action plan.

Establishing Phase:

During the Establishing phase, the issues that the organization has decided to address with its improvement activities are prioritized. Strategies for pursuing the solutions are also developed. The draft action plan is completed in accordance with the organization's vision, strategic business plan, lessons learned from past improvement efforts, key business issues facing the organization, and long-range goals.

During the Establishing phase, measurable goals are developed from the general goals that were defined in the Initiating phase; these measurable goals will be included in the final version of the action plan. Metrics necessary to monitor progress are also defined, resources are committed and training provided for the technical working groups or process action teams. The action plan developed will guide the improvement activities as it addresses the prioritized findings and recommendations from the Diagnosing phase. Also during this phase, tactical action plan templates are created and made available for the process action teams to complete and follow.

Acting Phase:

In the Acting phase of the IDEAL^SM model, solutions to address the areas for improvement discovered during the Diagnosing phase are created, piloted and deployed throughout the organization. Plans are developed to test and evaluate the new or improved processes. After the successful piloting of the new processes, and determining their readiness for organization-wide adoption, deployment and Institutionalization, plans to accomplish the rollout are then developed and executed.

Leveraging Phase:

The objective of the Leveraging phase is to make the next pass through the IDEAL^SM model more effective. By this time, solutions have been developed, lessons have been learned, and metrics on performance and goal achievement have been collected. These artifacts are added to the process database that will become a source of information for personnel involved in the next pass through the model. Using this collected information, an evaluation of the strategy, methods and infrastructure used in the improvement program can be performed and corrections or adjustments can be made prior to the start of another process improvement cycle. Some questions that should be asked include:

• Has the infrastructure (management steering group, process group, process action teams, etc.) performance been appropriate?
• Have the methods employed by the process action teams in their solution development activities been satisfactory?
• Have the program communications activities been sufficient?
• Does the sponsorship need to be reaffirmed?
• Does another baseline activity need to be performed?

The re-entry point into the IDEAL^SM model for the next cycle is highly dependent upon the answers to questions such as these.

---

Footnotes:

1. Charting the Seas of Information Technology, Chaos, Standish Group International, Inc., 1994.

2. The Project Sponsor is responsible for realizing the benefits predicted for the project and is typically a senior official responsible for the business function that the project will support.

3.. The Project Leader has overall responsibility for the project, is accountable for all internal and external aspects of the project and is typically a senior departmental official.

4. The Project Manager performs the day-to-day management of the project.

5. 'Return on Investment from Software Process Improvement as Measured by U.S. Industry,' Brodman, Judith and Johnson, Donna, Software Process, Improvement and Practices, 1995.

6. Charting the Seas of Information Technology, Chaos, Standish Group International, Inc., 1994.

7. Report of the Independent Review Panel on Modernization of Comptrollership in the Government of Canada.

8. Moving on Up: Data and Experience Doing CMM-Based Process Improvement, Hayes, Will and Zubrow, Dave, Software Engineering Institute, Carnegie Mellon University.

9. IDEAL^SM: A User's Guide for Software Process Improvement, CMU/SEI, 1996.