TBS Paper 
Date: 04-05-95
Subject: Potential synergy between review and quality management
Government Review and Quality Services Division
Financial & Information Management Branch
Treasury Board Secretariat
April 5, 1995
Introduction
This paper is addressed to federal review and quality management (QM) practitioners. It clarifies the concepts and practices involved in QM and review and documents lessons learned about their strengths and complementarity. It discusses opportunities for both, based on lessons learned from their strengths. It explores how intentional and visible collaboration between these functional groups can help both succeed in a more timely and cost-effective manner. The paper draws on discussions within the review and QM communities concerning direction, tactics and desirable partnerships. It is also based on Canadian experience in several jurisdictions and on developments in other OECD countries.

Quality management is the key to the government's managerial reforms. Having and acting on (a) evidence of how programs are working, (b) innovating to deliver better services to clients and (c) self-assessment by managers and work groups are key dimensions of the QM approach to public sector reform. They are also central features of the Treasury Board policy on review (Treasury Board, 1994). QM initiatives need the discipline and insights derived from review to succeed while review can best thrive in a quality-focused environment. Although review activities are sometimes seen as separate from quality concerns, review is best understood as a vital and integral part of the quality approach to the reform of public management. Indeed, review provides QM with a disciplined foundation.

In other OECD countries such as Australia, implementation of public service reform has been found to require strong linkages among its key strategic elements, "making performance count, leadership and strengthening the culture of continuous improvement" (MAB-MIAP, Australia, Publication No. 12, 1993). In the United States, long-term success in implementing the National Performance Review has been found to require government "to focus on results and avoid having the reform spin off in scores of different unconnected directions" (National Performance Review, 1993; Brookings Institution Center for Public Management, 1994).
1. Quality management concepts and practices
Quality management is a broad scope approach to improving the performance of public and private organizations. Its pioneers were Deming, Juran and Feigenbaum, Japanese theorists such as Ishikawa, Tagushi and Shingo and more recent gurus such as Crosby and Peters. Useful accounts of QM are found in Treasury Board Secretariat (1994b), Federal Interdepartmental Quality Network/Treasury Board (1992), Milakovitch (1995), Roston (1992, 1993, 1994), Clemmer (1992, 1995), Paton & Jelking (1994), Morgan & Murgatroyd (1994) and Éthier (1994).

The following images of QM are evocative for review practitioners:
•	"Total quality ... is an organizational policy which encourages continuing mobilization of all employees to improve (a) the quality of products and services, (b) the quality of the organization's operations and (c) the quality of its goals, in relation to the evolution of its environment." (Éthier, 1994)
•	"TQM is user-driven, ... concerned chiefly with changing attitudes and skills and a philosophy and culture of never ending improvement ... leading to customer satisfaction and then to customer delight". (Morgan & Murgatroyd, 1994)
•	"TQM is a total organizational approach for meeting customer needs and expectations that involves all managers and employees in using quantitative methods to improve continuously the organization's processes, products and services." (United States Office of Management and Budget Circular, 1990)
•	"The 12 attributes of the quality revolution are: management obsession with quality, passionate systems, measurement of quality, rewards and recognition for quality, everyone trained for quality, multifunctional teams, 'small is beautiful', creation of endless 'Hawthorne effects', an organizational structure devoted to quality improvement, involvement by everyone, 'when quality goes up, costs go down' and quality improvement is a never ending journey." (Peters, 1989)
The National Quality Institute (NQI) is a Canadian not-for-profit organization "committed to enhancing Canada's national well-being and global leadership through the adoption of total quality in the public, private, education, health care and labour sectors of (the) economy." (NQI, 1994) The NQI Quality Criteria and Principles define the quality approach to organizational well-being.

Underlying the Criteria are the following nine Quality Principles:
P1 cooperation, teamwork and partnering
P2 leadership through involvement and by example
P3 primary focus on clients/customers
P4 respect for the individual and encouragement for people to develop their full potential
P5 contribution of each and every individual
P6 process-oriented and a prevention strategy
P7 continuous improvement of methods and outcomes
P8 factual approach to decision making
P9 obligations to and expectations of stakeholders, including an exemplary concern for responsibility to society
The six NQI Quality Criteria are listed below, accompanied by the maximum number of points (total 1000) awardable in connection with each in organizational applications for the Quality Award, one of three awards under the banner of the Canada Awards for Excellence. (Other QM awards are the (Japanese) Deming Award, the (US) private sector Malcolm Baldridge National Quality Award, the (US) public sector Federal Quality Improvement Award and Presidential Award for Quality.)
C1 Leadership: strategic direction, leadership involvement (170)
C2 Client/customer focus: voice of the client, management of client relationships, measurement of client satisfaction (200)
C3 Planning for improvement: development and content of improvement plan; assessment (130)
C4 People focus: human resource planning, participatory environment; continuous learning environment; employee satisfaction (200)
C5 Process optimization: process definition; process control; process improvement (200)
C6 Supplier focus: supplier focus: partnering (100)
Each of the Quality Criteria contains an account of generic outcomes or success criteria that are expected to flow both within and outside the organization from attention to that criterion.

Numerous authors have discussed adaptations of the generic QM approach, developed in the private sector, necessary to make it applicable in various part of the public and parapublic (e.g., education, health, social services) sectors. Morgan & Murgatroyd (1994) distinguish among:
•	"core" QM concepts, tools and applications which apply everywhere, e.g., vision, strategy, measurable goals, empowered work teams, data- based decision making);
•	"adaptive" ones which can be interpreted to fit most occupational settings, e.g., client satisfaction, self-managing work-teams, just- in-time inventory management, partnering with suppliers); and
•	"problematic" ones which are appropriate to only some public and parapublic sector occupational settings, e.g., benchmarking and "zero defections".
2. Federal review concepts and practices
The approach of federal review - evaluation, internal audit, performance measurement, policy and regulatory review et al - to public service reform is essentially a subset of that of QM, although review developed historically in a different milieu (McQueen, 1992). The 1994 Treasury Board Review Policy aims to ensure "that the government has timely, relevant and evidence-based information on the performance of its policies, programs and operations, including the results they achieve and that it uses this information to improve management and cost-effectiveness and to account for results".
The Treasury Board policy on review stresses the following:
•	disciplined articulation of expectations for organizational performance 
o	modelling explicitly the goals and logic of an organizations's activities and environment, their expected impacts, the risks they face, needed controls (in the modern sense) and systems - taking seriously the interests and views of clients and other stakeholders and broader policy, risk management and horizontal issues
•	management by fact 
o	having and using timely and relevant evidence and analysis on whether policies, programs and operations are actually working as intended and on change in the internal and external environment - relying on disciplined internal audit and evaluation, performance monitoring, self-assessment and other review methods - supporting continuous individual and organizational learning
•	rethinking and improving policies, programs and operations and reporting on performance to Canadians 
o	delivering higher value, more appropriate and more cost- effective services to Canadians - using continuously improved design and delivery innovations and, sometimes, quite different tools and approaches (re- engineering) - credibly demonstrating and reporting on performance to all stakeholders
3. Harmonization of the key strengths of QM and federal review
Review is best seen as an important subset of QM, providing it with a disciplined foundation. QM initiatives need the discipline and insights derived from review to succeed, while review thrives best in a quality- focused environment. Disciplined articulation of expectations for organizational performance - the first element of review - reinforces the quality concern with organizational vision, leadership and the client focus. On the other hand, the quality focus on process optimization reinforces review's (third) concern with rethinking and improving services for Canadians.

The principal strength of review - management by fact - complements the key strengths of QM - organizational vision, leadership, programs driven by clients, continuous improvement and a strong people focus. In the same vein, Meyers & Hood (1994) of the Office of the Auditor General have documented a variety of ways of harmonizing "control and accountability with empowerment and innovation" in contemporary learning organizations:
•	"Organizations (have) moved from control to commitment, emphasizing results .."
•	"To stay relevant and effective during changing times, an organization must learn at least as fast as its environment changes."
•	"Empowerment involves equipping all staff to make decisions that will produce desired results."
4. Opportunities for QM based on lessons learned about federal review
Experience with review in Canada and other countries can contribute invaluable lessons for implementing broad-scope QM ideas and practices in the federal public service.

Review provides managers and other staff with robust tools for rethinking and reshaping activities and practices. Review avoids the trap of "empty empowerment". Numerous successes in rethinking program approach and delivery have been achieved by the review process in recent years:
•	performance frameworks: e.g., National Research Council, IRAP, Corporate Planing and Evaluation; Industry and Science Canada, Technology Outreach, Consumer Products, NAFTA Secretariat et al 
o	self-assessment against management control and accountability frameworks: Revenue Canada, National Capital Commission, Export Development Corporation, Veterans Affairs et al
o	The review community has achieved much success in discussing performance issues and the dynamics of change with politicians, managers, front line staff and stakeholder groups, including the vital linkages among performance issues of major concern to different groups. Review has taught us that partnerships among internal and external stakeholders around performance issues are essential for developing insight and consensus and for mobilizing energy for needed change.
•	Senge (1991, 1994) positions the development and use of a variety of "mental models" as one of the five key disciplines of continuous learning within organizations. This idea has also been systematically exploited by Morgan (1986, 1993) and others, using the idea of multiple "metaphors" for understanding organizational development. Review emphasizes the usefulness of multiple lines of evidence, based on models and metaphors, to develop value-added solutions to concrete performance problems.
•	Leadership and judgements by political and managerial leaders on resource reallocation have sometimes been found to depend crucially on fact-based review information. Indeed in some cases, review may be the only source of overall judgements as to whether results are good enough.
Review practice reinforces the QM reform agenda in other ways, for example:
•	Selecting and using key performance measures and service standards, a forte of review, to focus on customer and stakeholder interests and expectations.
•	Using a variety of review tools to help clarify key organizational relationships and, using the evidence gathered, to help decide whether incremental change or more fundamental re-engineering is in order.
•	Using solid fact-base information derived from review tools to strengthen a culture of continuous improvement.
•	Making the most of opportunities to use review tools to shape "organizations in transition" (Larson and Prestkill, 1991).
5. Opportunities for review flowing from lessons learned about QM
Recent accounts of the future of evaluation, internal audit and performance measurement in the professional literature have advocated much increased integration of review practices with mainstream innovation and re-invention concerns, e.g., Mayne (1994), Mayne & Zapico (1994), OCG (1991), OECD (1994ab), Ratliff & Beckstead (1994), Scriven (1993), Smith (1994), TBS (1994ac), Wye and Sonnichsen (1992) and Wargo (1994). Some of the lessons learned about implementing broad scope QM can enhance the success of the more specific review initiatives.
•	The QM focus on organizational vision strengthens review practice by placing it within the organizational and management mainstream.
•	In complex organizations, more attention needs to be paid to positioning review initiatives at appropriate levels and locations and aligning specific reviews with a broad QM vision of reinvented governance and service delivery.
•	Tested QM principles can be applied to the review process itself, e.g., for the development of client- and stakeholder-focused service standards for review practice.
•	The QM focus on teamwork and on individual and organizational learning reinforces review's emphasis on innovation and change.
6. Conclusion
This paper has described the main features of review and quality management and has dumented lessons learned about their strengths and complementarity. It has shown how QM initiatives can benefit from the discipline and insights derived from review and how review can better thrive in a quality- focused environment. Collaboration between review and QM can help both succeed in a more timely and effective way.
References
1.	Brookings Center for Public Management (1994), Reinventing Government: Appraising the National Performance Review
2.	Clemmer J. (1992), Firing on all Cylinders: the Service/Quality System for High-Powered Corporate Performance, 2nd edition, Macmillan Canada
3.	Clemmer J. (1995), Pathways to Performance: a Guide to Transforming Yourself, Your Team and Your Organization, Macmillan?
4.	Éthier G. (1994), La qualité totale: nouvelle panacée du secteur public?, Presses de l'Université du Québec, Ste Foy
5.	Federal Interdepartmental Quality Network/Treasury Board (1992), A Guide to Quality Management
6.	Larson C.L. & Preskill H. (eds) (1991), New Directions for Program Evaluation, 49, "Organizations in Transition: Opportunities and Challenges for Evaluation"
7.	Management Advisory Board-Management Improvement Advisory Committee, Australia (1993), Building a Better Public Service, Publication no 12
8.	Mayne J. (1994), "Utilizing Evaluation in Organizations: the Balancing Act", in Leeuw, F., Rist R. & Sonnichsen, R.C. (eds), Can Governments Learn? Comparative Perspectives on Evaluation and Learning, Transactions, New Brunswick
9.	Mayne J & Zapico E (eds) (1994) , Performance Monitoring for Public Sector Reform: Future Directions from International Experience, IIAA & Transactions, New Brunswick
10.	McQueen C. (1992), "Organizing for Program Evaluation in the Canadian Federal Government", ch. 2, Hudson J., Mayne, J., & Thomilson, R. (eds), Action-Oriented Evaluation in Organizations: Canadian Practices, Wall and Emerson, Toronto
11.	Meyers D. L. & Hood J. (1994), Opinions, 12 (2), 14-17, "Harmonizing Control and Accountability with Empowerment"
12.	Milakovitch M.E. (1995), Improving Service Quality: Achievinbg High Performance in the Public and Private Sectors, St Lucie Press
13.	Morgan C. & Murgatroyd S. (1994), Total Quality Management in the Public Sector: an International Perspective, Open University Press, Buckingham
14.	Morgan G. (1986), Images of Organization, Sage, Newbury Park
15.	Morgan G. (1993), Imaginization, Doubleday
16.	National Quality Institute (1994), A Roadmap to Quality for Business, Education, Government, Health Care, including Quality Criteria, Quality Principles and 10 Point Quality Checklist
17.	National Performance Review (1993), From Red Tape to Results: Creating a Government that Works Better and Costs Less, September 1993, Us Government Printing Office
18.	OECD (1994a), Service Quality Initiatives in OECD Member Countries, PUMA
19.	OECD (1994b), Performance Management in Government: Performance Measurement and Results-Oriented Management, Public Management Occasional Papers, no 3, PUMA
20.	Office of the Comptroller General (1991), Line Managers and Asssessing Service to the Public
21.	Paton R. & Jelking A. (1994), Optimum, 25(1), 35-41, "No-Name Management for the 90s"
22.	Peters, T.J. (1994), The Tom Peters Seminar: Crazy Times Call for Crazy Organizations, Vintage, New York
23.	Ratliff R.L. & Beckstead S.M. (1994), Internal Auditor, Dec 1994, 38-45, "How world class management is changing internal auditing", The Institute of Internal Auditors
24.	Roston, G. (1992), Beyond Slogans: Quality Management in Public Administration, Consulting and Audit Canada
25.	Roston G. (1993), The Public Manager, 53-56, Fall, "Quality and Competitiveness: New Partnerships, New Structures"
26.	Roston G. (1994), "Getting the Public Service Right: a Canadian Government Perspective", Presentation at the Brookings Institution Conference for US Federal Managers and Executives, 'Quality Management and Innovative Bratices in Business', Chicago
27.	Scriven M. (1993), "Hard-Won Lessons in Program Evaluation", New Directions for Program Evaluation, 5
28.	Smith M.F. (ed) (1994), Evaluation Practice, 15(3), on "Past, Present and Future Assessments of the Field of Evaluation"
29.	Senge P et al (1994), The Fifth Dimension Handbook, Currency/Doubleday
30.	Senge P. (1991), The Fifth Discipline: The Art and Practice of the Learning Organization, Currency (Doubleday), New York
31.	Treasury Board (1994), Policy on Review, Internal Audit and Evaluation, May 26, 1994
32.	Treasury Board Secretariat (1994a), Blueprint for Renewing Government Services Using Information Technology, Financial and Managemnt Information Branch
33.	Treasury Board Secretariat (1994b), An Overview of Quality and Affordable Service for Canadians: Establishing Service Standards in the Federal Government, including a draft of "A Declaration of Quality Service"
34.	Treasury Board Secretariat (1994c), The People Side of Reengineering, Human Resources Policy Branch
35.	Wargo M.J. (1994), Evaluation Practice, 15 (1), 63-72, "The Impact of the President's Reinvention Plan on Evaluation"
36.	Wye C.G. & Sonnichsen (eds) (1992), New Directions for Program Evaluation, 55, "Evaluation in the Federal Government"