Treasury Board of Canada Secretariat
www.tbs-sct.gc.ca
Common menu bar links
Breadcrumb Trail
ARCHIVED - Using External Service Delivery Key Performance Indicators
This page has been archived.
Archived Content
Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.
MAF Category: Citizen-Focussed Service
Theme: Quality Measures
Quality measures are indicators that provide analytical evidence in response to questions such as the following: Did the visitor complete the service transaction with the agent successfully? Was the interaction delivered with efficiency, clarity, and sensitivity? This type of subjective assessment of a service call requires evaluation by a supervisor by watching the interaction.
Another method used to collect quality information is the survey. GoC departments and agencies are directed to assess client satisfaction annually. Most commonly, surveys are conducted on a regular schedule using external customer satisfaction measurement suppliers. The Common Measurements Tool, or CMT (see http://www.iccs-isac.org/eng/cmt-about.htm), developed by the Institute for Citizen-Centred Service is the de facto standard for public-sector client satisfaction measurement. Employment Insurance offices have had some success with kiosk-style survey machines placed along exit paths in some in-person offices.
A third method that is not commonly used in the public sector but may prove valuable in the future is the "mystery shopper." A company is hired to place professional customer service analysts in specific in-person service programs. The service experience, conducted without the knowledge of the agent, is documented carefully and analyzed within the context of multiple evaluations. This particular type of measurement process is expensive and requires careful management of the employer‑employee relationship.
The GoC KPI working group proposed four measures for quality. Answer Accuracy requires either a post-visit survey to be completed by clients or supervisor assessment of the service interaction. Professionalism is one of the criteria assessed using the CMT. It includes highly valued factors such as vocabulary, courtesy, and sensitivity. Transaction Duration Variability has been proposed as a measure that correlates with quality. However, the working group has not analyzed this measure in detail. Finally, Critical Error Rate counts those transactions accepted at the counter that cannot be completed due to missing or erroneous information. This measure, primarily useful for those services that are high volume and transaction-based, can be tracked accurately within most of the "back-office" transaction processing systems. Readers should note that pending further analysis and definitional work, the working group has not yet recommended any of the above as a core measure.
Answer Accuracy and Professionalism as measures are not directly supported by the service management technologies. Transaction Duration Variability and Critical Error Rate are supported.
Transaction Duration VariabilityDuration of service at the service window categorized by transaction type |
Vendor Specifics (Q-MATIC)
Transaction Time reports are available for daily, weekly, and annual reporting periods. Figure 6‑3 shows an example of the Daily Transaction report with the distribution of Transaction Duration. For a statistical analysis of variance calculation, the duration row would be exported to an appropriate spreadsheet or analysis tool.
Figure 6-3: Transaction Duration Report (Q-MATIC)
Critical Error RateCount of specific service outcomes where service could not be completed due to the incomplete or inaccurate provision of prerequisite information divided by the total number of applications processed in the reporting period |
Vendor Specifics (Q-MATIC)
Most service management products provide some capability to record a result of a service visit. Result codes can be used to categorize transactions, errors, callbacks, or any other output of the program requiring tracking. Q-MATIC provides this capability through a feature called "Matter Codes." Figure 6-4 shows a sample Matter Code report.
Figure 6-4: Matter Code Report (Daily) for
Tracking Error Rates (Q-MATIC)
