Performance Measurement for the Government On-Line Initiative

October 2004

© Her Majesty the Queen in Right of Canada, 
represented by the President of the Treasury Board, 2004

This document is available in alternative formats
and on the Treasury Board of Canada Secretariat’s Web site at the following address:
www.tbs-sct.gc.ca

Table of Contents
Executive Summary
1) Introduction
2) Performance Measurement Methodology
The measurement regime
Data sources
3) Logic Model
Diagram
Gateways and integrated Web portals
On-line services
A common infrastructure
Leadership and direction
4) Government On-Line Results (2003)
Summary of results achieved
Convenience
Accessibility
Credibility
Critical Mass of Services
Service Transformation
Client Take-up
Client Satisfaction
Security
Privacy
Efficiency
Innovation
Appendix #1: Expected Outcomes
Appendix #2: Service Maturity Model
Appendix #3: Transformation Model

Executive Summary
Through Canada’s Government On-Line (GOL) initiative, the most commonly used federal information and transaction services will be available on-line by the end of 2005. The approach GOL has taken is citizen- and client-centred, as well as “whole of government”; the overall objective is to put federal services on-line in a manner that increases client satisfaction with services, while also improving the operational efficiency of service delivery.
This report measures progress towards expected outcomes for the GOL initiative, including:
4	increased citizen and client focus through convenient on-line service delivery, accessibility for persons with disabilities and in both official languages, and the provision of credible and reliable information;
4	better service quality through the availability of a critical mass of on-line services, robust take-up of and satisfaction with on-line services, and the transformation (or rethinking) of services from the citizen’s and client’s perspective;
4	increased trust and confidence in on-line service delivery through good security and privacy safeguards; and
4	positive economic impacts through greater efficiency in service delivery and innovative services.
Data sources to assess performance across these eleven dimensions include: citizen and client feedback from a range of surveys, departmental self-assessments of GOL progress, and third-party analysis and benchmarking.
In 2003, the GOL initiative remained on track in terms of putting the 130 most commonly used federal services on-line, and developing their target levels of functionality. The achievement of these target levels of functionality will be the key measure of success for the Government of Canada’s commitment to put the most commonly used federal services on-line by the end of 2005.
The Internet is now the second most frequently used method of contacting the federal government. A high percentage of Canadians think that on-line federal services are easy to access and use; a high percentage report that they could find the information they were seeking and that it was up-to-date. 
Canadians have more confidence in governments than either banks or the private sector in terms of offering safe on-line services. A reliable “whole of government” incident handling process is in place to respond to external electronic threats, and the Secure Channel – a comprehensive set of network infrastructure and operations services, security services, and enabling software tools – offers secure authentication and e-payment capabilities. 
While many Canadians still have concerns about conducting on-line transactions that require the exchange of personal information, a high percentage think that the federal government has put appropriate privacy safeguards in place. Privacy Impact Assessments are conducted before services are put on-line to ensure that designs safeguard personal information – more than 50 services have conducted an assessment (or preliminary assessment) since this requirement came into place in 2002, a number that is expected to double over the next two years.
For the most part, departments and agencies have automated services, achieving modest improvements in client convenience and processing requirements, rather than completely rethinking or integrating them. Fundamental service transformation is more complex, risky, and costly; because of this, departments and agencies have been encouraged to determine the appropriate degree of transformation for each service. Progress is being made in laying the foundation for fundamental service transformation – for example, through the development of inter-departmental gateways and Web portals, a common secure infrastructure that supports interoperability, and integrated service delivery pilots. A key challenge in 2004 will be to define what a longer-term, fundamental transformation agenda would look like, including the principles that would guide this effort, measurable and time-limited goals, as well as indicators that would track progress towards their achievement.
Also in 2004, departments and agencies will continue their efforts to improve the navigation and format of Web sites, accessibility for persons with disabilities, and the quality of French language content. Canadians’ awareness of on-line federal services remains modest – a targeted marketing strategy is needed, as well as channel migration and management strategies in order to encourage clients who are willing and able to move to self-service channels; greater take-up of on-line offerings will maximise departmental productivity, cost savings, and cost avoidance benefits. Departments and agencies also need to regularly conduct client satisfaction surveys, publish the results, and use them to establish service improvement strategies.
The full results from the GOL initiative will be realised over time. The federal government will continue to refine its performance measurement approach in order to strengthen both the reporting of progress as well as the use of this data to improve service delivery to Canadians.

1) Introduction
Through Canada’s Government On-Line (GOL) initiative, the most commonly used federal information and transaction services will be available on-line by the end of 2005. 
The approach the GOL initiative has taken is both citizen-/client-centred and “whole of government”; its overall objective is to put federal services on-line in a manner that increases satisfaction with services, while also improving the operational efficiency of service delivery. The following report summarises the specific outcomes that are expected, the results achieved to date, and the actions needed to improve performance and strengthen the measurement regime. 
The emphasis on outcomes is a reflection of the fact that four years have passed since the 1999 Speech from the Throne announcing the GOL initiative. The most commonly used federal services are now at least partially on-line. Because of this, it is appropriate to focus on the impacts – or results – of these services, including the extent to which clients are using them, their perceptions of service delivery quality, the extent of interjurisdictional collaboration, and efficiencies resulting from electronic service delivery. 
2) Performance Measurement Methodology
Measuring progress towards expected outcomes has been an important part of the GOL initiative since its inception. Initially, GOL departments and agencies were required to create an e-mail address that clients could use (one-way communication), to put information about key services on-line, to make key forms available electronically, and to establish a basic search capability on their Web sites. Departments and agencies reported at the end of 2000 that they had met these targets.
In addition, all federal institutions subject to Common Look and Feel Standards for the Internet (CLF) – a group that includes, but which is broader than, GOL departments and agencies – were required to implement these standards by the end of 2002, making their sites easy to recognise and navigate, ensuring equal access for persons with disabilities and in both official languages, and standardising the placement and content of important notices such as privacy and copyright. Using a self-assessment tool, in 2003 federal institutions reported on progress made in implementing CLF. The use of this assessment tool raised questions about its methodology, making it difficult to understand the current overall level of compliance with CLF. These questions will be resolved prior to the next reporting cycle. The aggregate results dealing with convenience, accessibility, and privacy are found in Section Four of this report.
A set of expected outcomes was also established for electronic service delivery (ESD) in 2000, emphasising greater functionality:
4	key federal services available on-line, with apply, file, and enquire capability;
4	an electronic payments capability;
4	a secure transaction capability;
4	access to integrated information and services through Web portals;
4	an advanced search capability;
4	two-way e-mail communication (or an appropriate alternative when it is necessary to provide a secure exchange of information);
4	greater personalisation available through portals;
4	enhanced information management principles in place and in use;
4	interjurisdictional ESD pilots; and 
4	testing of leading-edge technologies to improve and integrate service delivery.
GOL departments and agencies were required to achieve these outcomes by the end of 2005. 
The measurement regime
As experience with on-line service delivery was acquired from 2000 onward, a broader range of outcomes expected over a longer horizon was defined, focusing increasingly on the results for citizens/clients and the benefits for the federal government, such as client take-up of on-line services, higher levels of user satisfaction with service delivery, and cost savings or efficiencies as a result of ESD. 
This approach was consistent with what other jurisdictions were learning, as well as what external performance measurement experts working in this field had recommended. For example, Gartner argued that 
government organisations need a much wider range of metrics…Instead of measuring success based on rankings in Web service availability studies, governments should focus on metrics that demonstrate operational efficiency and value to constituents (“The Gartner Framework for e-Government Strategy Assessment”, Strategy Analysis Report, 2002, p. 5).
In 2002, therefore, the Treasury Board of Canada Secretariat (TBS) collaborated with Accenture, a multinational consulting firm with an extensive e-government practice, and a group of GOL departments and agencies to develop a more results-based measurement regime. The first step was a review of existing literature on e-government performance measurement. Overall, the findings were that this was a relatively new area, and that little work had yet been done. Nevertheless, the review did identify two good examples from which Canada could learn: Singapore and Ontario. Singapore uses an on-line self-assessment questionnaire to measure e‑government progress in multiple areas. Ontario includes client satisfaction as one of the key expected outcomes of e-government, which (like the Government of Canada) it assesses directly with clients using a standardised methodology, the Common Measurements Tool (CMT). The environmental scan also incorporated learning from other jurisdictions, including a recent report from the National Audit Office in the United Kingdom, which argued for more emphasis on monitoring the take-up and marketing of on-line services (Better Public Services Through e‑Government, 2002).
The second step was to establish the principles – a foundation – from which the GOL initiative could evolve its measurement regime:
4	develop a broad perspective of success;
4	recognise that ESD is one of several delivery channels that departments and agencies use;
4	recognise that developing on-line services is only an intermediary step;
4	ensure that the approach taken is flexible; and
4	make reporting easy to implement, and useful for departments and agencies.
Perhaps the most important of these principles is developing a broad perspective of success. This is because the number of services available on-line does not matter if clients have difficulties in accessing them; and similarly, the sophistication of these services is not relevant if clients do not like and do not use them. Another important principle is recognising that the development of on-line services is only an intermediary step towards a larger goal. This is to say, GOL is not simply about meeting a deadline; rather, it is about enabling departments and agencies to better achieve their mandated objectives. Further, it is necessary to ensure that the approach taken is flexible. Measurement strategies need to recognise, and account for, the significant differences that exist among departments and agencies – those offering primarily information services versus those offering large transaction services, and those providing benefits versus those ensuring regulatory compliance.
With this foundation, a set of shared expected outcomes was established:
4	increased citizen/client focus through convenient on-line service delivery, accessibility for persons with disabilities and in both official languages, and the provision of credible and reliable information;
4	better service quality through a critical mass of on-line services and transformation (or rethinking) of services from the citizen’s/client’s perspective, as well as greater take-up of, and satisfaction with, these services;
4	increased trust and confidence in on-line service delivery through good security and privacy safeguards; and 
4	positive economic impacts through greater efficiency in service delivery and innovative services. 
These expected outcomes are consistent with the “commitments to Canadians” made in the first overview report on the GOL initiative. They also incorporate the original set of expected outcomes articulated in 2000 (see Appendix #1).
The final step was to determine the techniques that would be used to assess progress towards these expected outcomes, and the levels at which they would be assessed. In terms of technique, the GOL measurement regime combines self-assessment data from departments and agencies with citizen/client feedback. Self-assessments allow the individuals who are closest to, and thus have the clearest understanding of, the on-line services being developed to measure their progress. They are also relatively inexpensive and quick to do. Even so, departments and agencies sometimes have a different understanding than their clients of, for example, the benefits of a particular on-line service. There are also aspects of e-government that they are not in a position to accurately assess, such as confidence in transacting securely on-line. For these reasons, it is necessary to balance departmental self-assessments with data from citizen/client surveys as well as focus groups. This use of multiple types of data also ensures a full and robust gauge of results achieved to date.
In terms of level of measurement, the focus is on services, departments and agencies, and/or the “whole of government”, as appropriate. For example, client take-up is always measured at the service level; implementation of CLF is a departmental way of improving recognition and accessibility of Web sites, and thus should be measured at the departmental level; integrated Web portals are a “whole of government” tool to improve convenience, and thus should be measured at that level.
Data sources
A key element of successful performance measurement is a data collection process that is both easy to implement, and useful for departments and agencies. To the extent possible, therefore, the GOL measurement regime relies on existing data sources. The following describes these data sources as well as some of the key assessment tools that are being used to assess the progress of the GOL initiative.
Departmental reporting on GOL plans and progress – Departments and agencies provide annual reports on progress to date in developing on-line services and integrated Web sites and portals. For example, in terms of on-line services they provide information on project milestones and deliverables, take-up, client benefits and satisfaction ratings, partnerships, departmental benefits, infrastructure needs, communications and marketing plans, as well as the implementation of select policies. Standardised tools to help departments and agencies with their assessments include:
4	a three-stage, nine-point model measuring the increasing sophistication both of transaction services – from publish, to interact, to transact – and information services – from publish (basic information is available on-line), to customise (holdings are increasingly interactive), to provide client-defined access (users can increasingly manipulate/synthesise information from different sources);
4	a four-stage model measuring the degree of service transformation – from grouping related components together in order to facilitate access, to interoperability and some coordination across channels in order to personalise service, to integration of related offerings in order to offer bundled solutions (such as agreements with other organisations to work together, easy movement across delivery channels, and joined-up back-office systems), to a complete and seamless service across organisations and channels; and
4	the CMT, an interjurisdictional tool adopted by departments and agencies to measure client satisfaction across service delivery channels and across organisations.
Institutional reporting on CLF implementation – The target date for implementation of CLF was December 31, 2002. Federal institutions used a self-assessment tool in 2003 to review the progress made in implementing CLF, to identify any remaining barriers, and to set a target date for achieving full implementation (if applicable). Their summary reporting on certain CLF components – navigation and format, accessibility, official languages, and important notices – is particularly relevant for this performance measurement exercise. In terms of the methodology or approach, each institution assessed either all or a representative sample of its Web sites against the various standards described in the CLF self-assessment guide. They assigned scores out of 100 for each CLF standard based on either their assessment of the progress thus far (e.g., towards having Web content in the plainest and simplest language possible), or the number of their Web pages/documents that were compliant (e.g., with HTML as the primary format). Institutional scores for each CLF component are a simple average of the standards belonging to it. The aggregate or “whole of government” scores reported in Section Four of this report are calculated in the same way; they represent the overall progress that federal government institutions subject to CLF have made to date, as opposed to the percentage of departments and agencies (or Web sites) that are fully compliant. (Note: because Web information is continuously being added and updated, it will be important to review the CLF progress of departments and agencies in the future.)
Citizen/client feedback – Departments and agencies that are involved in the development of gateways and integrated Web portals regularly evaluate their work using focus groups, surveys, and other citizen-/client-based research (in 2003, 26 such studies were done). In addition, TBS collects information on a range of issues – including ease of access and navigation, clarity of language, perceptions of privacy and security – through a GOL Internet Research Panel that it has established. TBS also monitors and, where appropriate, participates in public opinion research on ESD done by other government organisations and private sector consulting firms; research ranges from Statistics Canada’s (Stats. Can.’s) Household Internet Use Survey to EKOS’ Information Highway studies and the Institute for Citizen-Centred Service’s Citizens First reports. 
Other data sources – In addition to the above, the GOL measurement regime incorporates data from a number of other sources; for example,
4	Web metrics;
4	annual reports done by the Official Languages Commissioner and the Privacy Commissioner;
4	external indices such as Stats. Can.’s Survey of Electronic Commerce and Technology, the Economist’s E-Readiness Index, the World Economic Forum’s Networked Readiness Index; and
4	international comparisons of various e-government initiatives that are done by academic institutions like the Centre for Public Policy at Brown University, non-governmental organisations like the UN’s Division for Public Economics and Public Administration, and private sector consulting firms like Accenture.
3) Logic Model
The purpose of a logic model is to identify the causal linkages between the activities of a policy, program, or initiative, the outputs, and the achievement of its outcomes – to highlight the steps that would demonstrate progress towards the final goal, and to help determine where to focus measurement efforts (TBS, “Guide for the Development of Results-based Management and Accountability Frameworks”, 2001).
The following diagram and text describe the GOL logic model, showing the relationship between work on gateways and integrated Web portals, services, infrastructure, and policies, and the overall objective of increased satisfaction with federal services while at the same time improving the operational efficiency of the Government of Canada (although specific cost savings targets have not been set).

Diagram

Gateways and integrated Web portals
One of the most common barriers that clients encounter is not knowing which department or agency, or which level of government, is responsible for delivering which service. In response to this, the federal government, as part of the GOL initiative, has developed integrated Web sites and portals – single points of access to on-line information and services.
The short-term outcomes include increased convenience to clients of on-line federal information and services. Integrated Web portals are making services easier to find by organising information according to the client’s, not the government’s, perspective – that is, according to subject or audience – and by grouping related offerings together, beginning with the federal government but including services from other jurisdictions over time.
In the medium-term, integrated Web portals will help increase the take-up of on-line federal services by improving Canadians’ awareness of the range of on-line services available to them, and help promote more fundamental transformation of service delivery from the citizen’s/client’s perspective. When clients can easily find the services they want through the Internet, they will be more likely to use this delivery channel rather than traditional ones, such as the phone or an in-person visit. As well, rationalising grouped information will help eliminate overlap, potential conflicts, and gaps among information and service offerings, as well as promote the rethinking and integration of business processes. This transformation work will require new governance models, and will extend beyond the GOL timeframe.
On-line services
In addition to improving access, the GOL initiative is developing an on-line option for 130 of the most commonly used federal information and transaction services – those services that touch the lives of the greatest number of Canadians. 
The short-term outcomes are to increase the on-line availability of credible and relevant federal information, as well as a critical mass of services. Extensive information holdings increasingly are being made available through the Internet, either in support of on-line transactions or as services in their own right. As this happens, departments and agencies are upgrading processes to manage these holdings – to ensure that they are accurate and up-to-date – and are offering them in plain language formats. Putting the most commonly used federal services on-line is building and sustaining the momentum (the “critical mass”) for e-government. These services are easy and convenient to use, thus encouraging clients to use the Internet to interact with the federal government on a regular basis.
The medium-term outcomes of a critical mass of appropriately marketed on-line services will include increasing the client take-up of these services, ensuring that all individuals and businesses that want to use the Internet channel do so. As clients become aware of the benefits that on-line services offer – including more complete information, time savings, cost savings, simplified interactions – and as they are able to conduct an increasing variety of transactions with the federal government through the Internet, they will use on-line services more frequently and for more types of transactions. Further, continued client demands for service improvement and lessons learned through the GOL initiative will help build capacity within the federal government for more fundamental transformation of service offerings and service delivery in the medium- to longer-term. 
Fundamental service transformation – the complete rethinking of programs, business processes, and associated policies to achieve significant client benefits and federal savings – is not possible by the end of 2005. However, efforts to date in restructuring information, services, and Web sites according to client expectations provide a foundation and an impetus for more profound changes.
A common infrastructure
The GOL initiative is developing a “whole of government” Secure Channel to provide secure, responsive, and economical on-line access to the federal government, as well as common network, support, authentication, information management, electronic payment, and other services in response to department and agency needs.
The short-term outcomes are robust security and privacy protection for on-line federal services. The services currently provided by the Secure Channel guard against network intrusions, ensuring the integrity of Web sites and databases, facilitating the secure exchange of information both externally and internally, as well as enabling transactions through services such as secure electronic payments processing. The Secure Channel is one of the ways of providing better security than various alternatives, such as Secure Socket Layer (SSL), because it includes: end-to-end encryption of data, a common solution for the identification and authentication of clients, and the infrastructure required for digital signatures (which verify not only identity, but also the authenticity of documents).
The medium-term outcomes will include increasing the take-up of on-line services and providing key enablers for more fundamental service transformation. First, the security and privacy safeguards provided through the Secure Channel will build citizen trust and confidence in using on-line federal services. Moreover, a common infrastructure like the Secure Channel will provide key enablers for “joined-up” and horizontal government services – it includes a telecommunications network for all federal departments and agencies that can be extended to other jurisdictions/third party organisations, and in the future will offer common applications facilitating the integration of related services. Migration to a common infrastructure will also enable new services and functionality (for example, the secure on-line viewing by a client of all government accounts) at a lower cost than if departments and agencies were to build their own infrastructures. 
In addition, the Secure Channel is an example of the innovation that the GOL initiative is achieving, as it is a dynamic public-private sector partnership developing and using world-class technology to improve government service delivery.
Leadership and direction
This component reflects the role of policies and strategies to direct and coordinate the federal presence on-line, and to guide the development of key aspects of on-line service delivery. The implementation of Common Look and Feel Standards for federal Web sites increases the convenience and accessibility of information and services: sites are easily recognisable through the use of Federal Identity Program identifiers, have common navigational features, and are designed to serve the needs of disabled persons as well as both official languages groups in Canada. Updated IT security standards provide direction for the security of on-line services. Privacy impact assessments ensure that services are only put on-line when they offer good privacy protection.
Skills development strategies support the development of a critical mass of services and will be necessary as a result of more fundamental transformation of service delivery. New skills and tools for both employees and managers will help enable the shift of service delivery to the Internet channel, but also to a multi-channel environment managed at the enterprise level. In addition, ESD and service improvement targets, enterprise architectures, and accountability frameworks will support citizen-/client-centred service and the sound management of IM/IT resources, as well as increase client satisfaction and the use of common services and applications, thus helping achieve the longer-term GOL objective of operational efficiency. 
4) Government On-Line Results (2003)
Summary of results achieved
Canada continues to do well in international benchmarks of progress in developing e‑government. According to the 2003 Accenture study, e-Government Leadership: Engaging the Customer, Canada ranked #1 in the world for the third year in a row, largely because of its work to date in transforming services. Noted strengths of the Canadian approach include: the development and evolution of on-line services in response to citizen/client demand, a focus on a coordinated multi-channel service delivery strategy, and the implementation of horizontal solutions such as Common Look and Feel Standards for the Internet and the Common Measurements Tool.
In a follow-up study, Accenture suggested that there was a gap between Canada’s current #1 ranking and the perceptions of its citizens (e-Government: The Citizen’s View, 2003). For example, in a comparison of Internet users in six different countries – Australia, Britain, Canada, Singapore, Spain, and the United States – Canadians ranked #5 in terms of knowledge of the benefits of on-line service delivery, suggesting that more targeted marketing and communications strategies are needed.
Brown University ranked Canada #3 in the world in 2003 (Global e-Government). It points to: the high level of development of the Canada Site, gateways, and integrated Web portals; easily comprehensible reports, guides, and other publications; the personalisation and customisation options available on federal Web sites; and the wide availability of on-line privacy notices. Using a slightly different methodology, the United Nations (UN) ranked Canada #3 in the world in terms of e-participation and #6 in terms of the maturity of on-line services (Global e‑Government Survey, 2003). For the UN, one of Canada’s strongest e-government features is its focus on developing portals.
These international benchmarks serve as a context for the results that the GOL initiative achieved in 2003, as assessed through its measurement regime. 
A citizen/client focus – The GOL initiative has improved the convenience of on-line federal information and services. A strong majority of Canadians who use the Internet think that these services are easy to access and use; they rate the Internet channel as more convenient than the telephone or in-person visits. Partly because of this, the use of the Internet is increasing in popularity over time, and is now the second most frequently used method of contacting the federal government. There are still challenges, however, in effectively organising on-line information and services so that access is intuitive for all users, and in improving the performance of search engines on federal Web sites. Efforts to enhance on-line offerings in ways that are consistent with user feedback will continue. In addition, the roles and performance expectations for gateways and integrated Web portals will be clarified, and a performance measurement regime will be developed and implemented to periodically assess them.
For the most part, federal Web sites provide information and services in both official languages, although the quality of French language content is occasionally less than desirable or inconsistent with the English equivalent. The majority of these sites also provide descriptive or alternative text for elements such as sound, videos, and images. Doing so improves access for persons with disabilities who rely on Web readers or other assistive technologies. Departments and agencies need to continue to work to ensure that French language content is of a high quality, and that there is equitable access for everyone who wants to use the Internet to interact with the federal government.
A high percentage of Internet users think that on-line federal information is up-to-date; they are more likely to think that it is the most current information available. User satisfaction with the amount of information on federal sites varies, but it is clear from the results of surveys that satisfaction increases with greater knowledge of federal Web sites – implying that site layouts can be improved – and there is a healthy demand for more information to be put on-line.
Service quality – The 30 departments and agencies participating in the GOL initiative are on track to meet the key GOL objective of putting the 130 most commonly used federal services on-line by the end of 2005. Each of these services has a target for the level of functionality that will be available for Internet users by the end of 2005. By the end of 2003, 35% of GOL services had reached their targeted level of functionality. By the end of 2005, the average information service will be at a “level 6” – it will provide users with a full range of interactive tools and navigational aids in order to access and customise extensive information holdings on-line. The average transaction service will be at a “level 7” – it will provide clients with the ability to at least partially complete a binding transaction with the federal government on-line (for example, submit an application on-line but provide supporting documents through another service delivery channel). 

Program managers are mostly automating existing services, and making modest improvements in client convenience and processing requirements. Few of them are planning more fundamental service transformation, as measured by a target “level 8-9” for information services (allowing clients to pull together information from different sources, resulting in a single information product with the appropriate level of detail) or “level 9” for transaction services (so clients can complete a transaction in real time without leaving the Web site), or as measured by the proportion of services which will provide complete and seamless service across all organisations and channels by the end of 2005.
GOL departments and agencies were free to determine the appropriate degree of service transformation for each service by the end of 2005. Fundamental service transformation means changing the form or structure of a service in order to achieve “order of magnitude” improvements in value to clients, and cost savings or other benefits to government. Compared with making incremental improvements, fundamental service transformation is much more difficult, risky, and costly. It also involves working across programs, departments and agencies, and jurisdictions to integrate services and move to common databases or back office systems. Doing this is much more complex and time consuming than improving single, siloed programs. It is for these reasons that the federal government has elected to take a cautious approach – to lay a foundation for fundamental service transformation through the development of a citizen-focused, enterprise-wide Web face of government as well as a common secure infrastructure that will support interoperability and the development of enterprise business and information architectures, and to learn from selective investments in strategic service transformation pilots and projects. Nonetheless, it is timely to establish a clear vision and outcomes for service transformation over the remaining period of the GOL initiative, as well as define objectives and outcomes for the following period.
Half of Canadians (66% of Internet users) report having visited a federal Web site at least once in the past year. However, reported awareness of the on-line services that the federal government offers is modest. Only half of GOL departments and agencies have plans to market on-line services in 2004; marketing plans only exist for 22% of services (and only for 13% of services that are complete in terms of having achieved their target level of functionality). A government-wide marketing strategy is needed, as is more effort by departments and agencies to increase the awareness of on-line services, particularly among Internet users. 
Visits to gateways and integrated Web portals are increasing – the most frequently visited portals are Business Start-up, Jobs, and Going to Canada; the fastest growing in terms of traffic volume are Canada and the World, Financing, and Public Safety. The number of on-line interactions (queries, browsing for information, etc.) between clients and GOL departments and agencies grew 54% between 2002 and 2003; the growth was mainly due to information services. The proportion of clients completing transactions on-line is also increasing. In 2003, 24% of GOL clients completed transactions on-line, up from 21% in 2002. Individual income tax filing and job searches drove a large part of this take-up – without these services, the percentage of clients completing transactions on-line was 12% in 2003, up from seven percent in 2002. One of the challenges in preparing this report was the varying quality of data. More work is needed to establish better metrics to understand client use of federal Web sites, and to obtain more consistent data on client interactions and transactions by service. Not all departments and agencies set client take-up targets for on-line offerings, and more aggressive marketing and migration strategies are needed in order to encourage clients to move to self‑service channels where this is their preference.
Client satisfaction with on-line offerings is moderately high, as can be seen from the increasing use of the Internet channel as well as public opinion and client surveys. There continues to be strong support among Canadians for the concept of organising Web sites according to the “no wrong door” principle, and the majority of users of federal sites report finding the information they were seeking. According to the 2003 Citizens First survey, Canadians’ satisfaction with federal services is 64%. By channel, they are more satisfied with government services available through the Internet channel (68%) than through other channels such as the telephone (56%). Consistent with this, the average satisfaction score for GOL services that have measured satisfaction specifically with the Internet channel is about four, or “satisfied”, on a five-point scale where one is “very dissatisfied” and five is “very satisfied”.
Improving client satisfaction with on-line services is a key objective of the federal government. The specific target is to increase overall satisfaction with service delivery by 10% by the end of 2005 (from 60% to 66%). In addition, all departments and agencies with citizen-facing services must publish satisfaction targets and standards for service delivery (such as timeliness), conduct client satisfaction surveys periodically, and publish results achieved. Robust measurement regimes will drive service improvement. Overall, however, current performance measurement and communication of results are weak – only half of GOL departments and agencies have surveyed clients during the past two years, and one-third have posted these results on their Web sites; likewise, about two-thirds of departments and agencies have set service standards, but only one-third have posted these standards on their sites. Departments and agencies will need to establish and publish targets as well as performance reports.
Trust and confidence in on-line service delivery – In general, Canadians have confidence in the security and privacy of on-line federal services; their confidence in federal Internet offerings is on par with, or higher than, their confidence in on-line banking services. More effort is needed to communicate with Canadians, both Internet and non-Internet users, about the security and privacy safeguards that are in place for on-line transactions and interactions, particularly as more services benefit from Secure Channel services such as on-line client authentication and electronic payments. Robust security measures are needed to protect the confidentiality and integrity of the information collected by the federal government. Departments and agencies are at various stages in implementing the Government Security Policy, but are meeting the security expectations of their clients; this policy requires threat and risk assessments and mitigation strategies, business continuity plans, compliance assessment, and enhanced training for all employees to foster a better understanding of security requirements and expectations.
Positive economic impacts – Some of the early benefits identified for clients of on-line services are easier access and time savings. Departments and agencies are also seeking benefits such as cost avoidance, cost savings and/or increased productivity. Some cost savings and/or avoidance are being realised in almost half of GOL services, and savings are being reinvested in the on-line channel and better service delivery. About two-thirds of GOL services are realising some productivity benefits, allowing employees to shift their focus from routine and low-value transactions to more complex client requests and higher-value activities. 
There is no evidence of significant savings from the shift to on-line services, however, and many would argue that this is because it will take time to realise savings, as the Internet is an additional channel and existing channels have not been closed. On the other hand, there is little evidence that GOL departments and agencies are setting explicit benefits targets for services. There is also a lack of reliable information on service delivery costs by channel, which, together with client migration strategies, is necessary for the effective management of service delivery across channels within departments and agencies, and at an enterprise level across the federal government. About two-fifths of GOL departments and agencies were unable in 2003 to predict the impact of the Internet channel on their in-person, call centres, and/or mail centres for at least one of their services. They will need to develop benefits capture, client migration, and channel management strategies that are aligned with the service visions for individuals, businesses, and international clients over the next year. 
The detailed results follow. As mentioned above, the strategy has been to rely on existing data sources where possible, which vary in terms of scope and methodology. It will therefore be necessary to continue to work on strengthening data quality while ensuring that the collection process remains easy to implement from a departmental perspective. Further work is also required to determine whether, and where, additional performance benchmarks are appropriate.
	Convenience	
Indicator(s)	A “no wrong door” approach
Federal Web sites are easily identifiable and easy to navigate
Relevant on-line information and services are put together in ways that make sense from a citizen/client perspective	
Measurement Level and Technique	Measurement primarily at the “whole of government” level using citizen/client feedback	
Primary tool(s)/data source(s)	1)	Omnibus surveys – includes, e.g., EKOS’ Information Highway studies, Ipsos-Reid research, NFO Interactive surveys, TBS GOL Internet Research Panel
2)	Departmental reporting on implementation of the Common Look and Feel Standards for the Internet (CLF) – federal institutions self-assessed their compliance with the seven CLF components (33 standards in total) using either all or a representative sample of their Web sites; the score for each component is an average of what all departments and agencies subject to CLF have reported; the focus, here, is on navigation and format (CLF component descriptions, related standards, and a self-assessment guide are available at: http://www.cio-dpi.gc.ca/clf-nsi/index_e.asp )
3)	Meta-analysis of the citizen-/client-based research done by gateways and portals	
Summary of results achieved in 2003	Mostly positive – A high percentage of Canadians think that on-line federal information and services are easy to access. In addition, most report being able to find the information they are seeking. However, there continue to be challenges in effectively organising content in multiple ways so that Internet users can intuitively use federal Web sites regardless of the path they choose. As well, Internet users have mixed feedback about the performance of search engines on federal sites. Work is ongoing in each of these areas. There is also a lack of good comparable data on the progress that gateways and portals are making, e.g., the functionality they offer to clients, client take-up, and satisfaction scores.	
Raw data	Federal Web sites – access/”no wrong door”
•	82% of Internet users who recently visited a federal Web site think that the site was easy (27%) or very easy (55%) to access (GOL Internet Research Panel, 2004)
•	30% used a bookmark, 29% an Internet search engine, 28% the URL, six percent a link on non-federal sites, and three percent a link in an e‑mail message
•	Canadians are more likely to think that it is easy to access federal services through the Internet than through any other delivery channel (EKOS, 2003)
•	81% of Canadians who recently used the Internet to access a federal service think that it was easy to do so
•	In contrast, 66% of those who used the mail think that it was easy, 64% who went to an office, and 58% who used the telephone	
	Convenience (cont’d)	
	•	76% of Internet users think that they should be able to obtain federal services from different departments and agencies through a single federal Web site, about the same as in 2001 (75%) (EKOS, 2003)
•	Canadians also express strong support for designing federal Web sites according to the “no wrong door” principle – for providing multiple ways to find content so that access is “intuitive” (TBS meta-analysis, 2003)
•	However, only 47% of Internet users are somewhat (41%) to very (six percent) satisfied with the way that topics and subjects are currently organised on federal Web sites (NFO Interactive, 2003)
•	Satisfaction appears to increase with greater knowledge of the sites; 52% with between two and five visits in the past year are satisfied, but only 31% with one visit
•	Canadians have also reported certain organisational problems with gateways and portals; e.g., different routes taken on a Web site do not lead to the same content, information is not organised logically, and duplication of links on the same Web page can be confusing (TBS meta-analysis, 2003)
•	In response, in 2003 the Canada Site refined its structure; e.g., based on empirical research, it added additional links and cross-references as appropriate, and streamlined navigation paths through consistent linking strategies
Federal Web sites – use/navigation
•	86% of Internet users who visited a federal Web site in the past year found the information they were looking for, compared with 88% in 2001 (EKOS, 2003)
•	83% of Internet users say that it is somewhat (49%) to very (34%) easy to find the FAQ page, and 77% say that it is somewhat (47%) to very (30%) easy to find the e-forms and services page, on the Canada Site (Ipsos‑Reid, 2003)
•	94% of Internet users say they have noticed the top menu bar on federal Web sites, and 79% say they have used it (GOL Internet Research Panel, 2004)
•	Of those who have used the top menu bar, 89% have used the search function, 50% the link to the Canada Site, 42% the “Contact Us” button, 24% the “Help” button, and 12% the language toggle
•	78% of Internet users think that federal Web pages load quickly, 78% that it was easy to print forms or documents, 63% that search engines work well, and 63% that the sites are visually appealing (GOL Internet Research Panel, 2004)
•	According to another survey, however, only 41% of Internet users are somewhat (34%) to very (seven percent) satisfied with federal search engines (NFO Interactive, 2003)	
	Convenience (cont’d)	
	Common Look and Feel Standards for the Internet (CLF) – navigation and format
•	In 1998, Treasury Board Ministers established additional Federal Identity Program (FIP) requirements aimed at strengthening the presence and visibility of the federal government in all of its activities. One of the requirements is the consistent application of FIP to electronic services, including all federal Internet/Intranet sites, products, and deliverables; this resulted in the development of eight CLF navigation and format standards, such as the application of common and institutional menus with standard navigation elements, the use of standard metadata elements to aid resource discovery, date indicators, and others
•	The average level of implementation of the navigation and format component across departments and agencies subject to CLF is 84%, where each standard accounts for one-eighth of the total implementation level	
Plans for improvement	•	Development and implementation of a common accountability framework and related performance measures for gateways and portals to better assess progress, both strengths and areas requiring improvement, as well as roles relative to department and agency Web sites
•	Work to improve search engine results, as well as to refine the scope of portals, develop a usability testing lab, and enhance resource discovery through the use of XML and metadata
•	Continued testing of Web sites to ensure that potential problems relating to ease of access and ease of use are identified and addressed	

	Accessibility	
Indicator(s)	On-line information and services are accessible to persons with disabilities
On-line information and services are available in both official languages	
Measurement Level and Technique	Measurement at the departmental level using self-assessments, and at the “whole of government” level using citizen/client feedback	
Primary tool(s)/data source(s)	1)	Departmental reporting on implementation of the Common Look and Feel Standards for the Internet (CLF) – federal institutions self-assessed their compliance with the seven CLF components (33 standards in total) using either all or a representative sample of their Web sites; the score for each component is an average of what all departments and agencies subject to CLF have reported; the focus, here, is on accessibility and official languages (CLF component descriptions, related standards, and a self-assessment guide are available at: http://www.cio-dpi.gc.ca/clf-nsi/index_e.asp )
2)	Omnibus surveys – includes, e.g., Ipsos-Reid research
3)	Meta-analysis of the citizen-/client-based research done by gateways and portals	
Summary of results achieved in 2003	Mixed – Almost all Canadians think that they can obtain service in either official language through the Internet at least most of the time; they think that they are more likely to receive this type of service through the Internet than through any other delivery channel. More generally, the federal government has made good progress in implementing CLF standards relating to accessibility and official languages, although work remains to be done in order to achieve 100% compliance with this policy. Other challenges include, e.g., providing TTY numbers on “Contact Us” Web pages and making all aspects of Web sites understandable through Web readers. One of the ways these challenges are being addressed is through the development of an Accessibility Domain Architecture, which offers guidance for departments and agencies in choosing technology that provides equitable access at the outset of projects.	
Raw data	Access for persons with disabilities
•	CLF accessibility standards ensure equitable access to content on federal Web sites; they cover persons with disabilities who rely on assistive devices – e.g., voice recognition software and text readers – to access on-line services, as well as individuals whose technology – whether older Web browsers and slow connections, or digital cell phone displays and PDAs – may otherwise limit what they can do 	
	Accessibility (cont’d)	
	•	For all departments and agencies subject to CLF, the average level of implementation of the four standards supporting accessibility (W3C Web Content Accessibility Guidelines: Priorities One and Two, document technologies, alternative formats, and text equivalents) is 83%, where each standard accounts for one-quarter of the total implementation level
•	Remaining challenges include commercial off-the-shelf applications and legacy data that are partially inaccessible to those relying on assistive devices; departments and agencies are encouraged to provide the information necessary to obtain alternative formats such as Braille, and to continue their conversion efforts as well as the search for and sharing of solutions that address accessibility challenges 
•	The experiences of portal development teams suggest that accessibility challenges can remain even when Web sites have implemented CLF, such as making sites fully understandable through text readers
•	Departments and agencies are encouraged to continuously review and test their Web sites
•	There are also independent studies and client surveys pointing to accessibility challenges; e.g., a 2002 study found few “Contact Us” pages on federal Web sites provide TTY access for individuals with hearing disabilities (Sinclair, Nicholson, and Associates, “New Doors to Access: Are They Open or Closed?”, 2002); other issues Internet users have raised include the small font sizes of some sites, making text difficult to read (TBS meta-analysis, 2003)
•	Departments and agencies are encouraged in the December 2002 CLF best-practice document to add TTY numbers to contact information
•	Also in response to some of these challenges, the Canada Site is constantly researching, testing, and making adjustments to its coding, text, and layout in order to provide greater support for users of assistive devices
•	The Persons With Disabilities portal (http://www.pwd-online.ca/en/home.jsp) provides integrated access to information and services for persons with disabilities, their caregivers, and their families; it was developed in accordance with the W3C’s Web Accessibility Initiative Guidelines (Version 1.0) and with CLF
•	Features include: multiple font sizes for text, multiple colour schemes to improve contrast, links that are highlighted when an Internet user drags a mouse over them, and alternative site navigation using a keyboard rather than a mouse
•	Current information content includes: Transport Canada’s (TC’s) “Access to Travel” (information on accessible travel and transportation, including city maps in Braille), Industry Canada’s (IC’s) “Assistive Technology” (information on adaptive computer technologies), Natural Resources Canada’s (NRCan.’s) “Mapping for the Visually Impaired”, and links to federal disability information
•	Future plans include: links to provinces and territories and other related Web sites	
	Accessibility (cont’d)	
	•	TBS is also developing an Accessibility Domain Architecture to provide a “whole of government” approach for the design of accessible IM/IT systems; this architecture includes a variety of “personas” that clarify the challenges experienced by persons with disabilities and recommend ways to address them at the outset, thus providing guidance for departments and agencies in choosing technology that both addresses the need for equitable access and meets specific business requirements
•	Personas include: Jane – no disabilities, Edith – a senior citizen, 
David – spinal injury preventing the use of hands, Roy – severe cerebral palsy, Roger – onset of Parkinson’s disease, Ruth – deaf, Frank – blind (does not read Braille), Mary – chronic progressive course of Multiple Sclerosis, Nancy – deaf-blind and does not speak, and Francis – a young war veteran with post-traumatic stress disorder
Access in both official languages
•	93% of Anglophone Canadians and 91% of Francophone Canadians think that they can obtain on-line federal services in either official language at least most of the time (Ipsos-Reid, 2003, also, e.g., TBS meta-analysis, 2003)
•	This compares with 91% of Anglophones and 87% of Francophones for regular mail services, 88% of Anglophones and 84% of Francophones for phone services, and 86% of Anglophones and 81% of Francophones for in‑person services
•	76% of Anglophone Canadians and 60% of Francophone Canadians think that they can obtain   on-line federal services in either official language all of the time
•	For all departments and agencies subject to CLF, the average level of implementation of the 10 standards supporting official languages (e.g., bilingual domain names and welcome pages, user ability to choose among the two official languages) is 92%, where each standard accounts for one‑tenth of the total implementation level
•	In focus groups some Internet users reported finding errors on the French versions of federal Web sites (TBS meta-analysis, 2003)
•	This is consistent with an earlier finding of the Official Languages Commissioner that “the quality of the content on government Web sites occasionally leaves something to be desired from one organisation to another” (French on the Internet, 2002)	
	Accessibility (cont’d)	
	•	However, in her 2002-03 Annual Report to Parliament, the Commissioner noted the work that TBS has done in improving the quality of French on federal Web sites, e.g., an inspection of Web sites to ensure that there is a proper balance between English and French content and to check the quality of the language used, and the launch of a pilot project to examine different methods of increasing the availability of specialised reference material and information in French
•	The Commissioner also highlighted some of key remaining challenges, e.g., the development of a policy for on-line publication of scientific and historical information in both official languages, and the publication of more specialised documents in French
•	Further, GOL departments and agencies report that senior managers and/or senior communications personnel review Web content to ensure it makes sense to clients, including from an official languages perspective
•	More detailed results can be found in the credibility section of this report	
Plans for improvement	•	Implement a strategy to encourage departments and agencies to achieve 100% compliance with CLF, including accessibility and official languages standards
•	Establish an Internet Advisory Committee Quality Assurance Working Group to revise the compliance measurement methodology
•	Increase small department and agency awareness of CLF requirements and best practices through presentations to the Small Agencies Administration Network and to committees such as the one for heads of IT for small departments and agencies
•	Encourage departments and agencies that have not yet established Centres of Expertise for the Internet to do so
•	Develop a communications strategy and products to promote and support CLF awareness and compliance
•	Hold a CLF and Centres of Expertise workshop to increase awareness and share information, tools, and best practices with the Web practitioner community
•	Review and propose changes for the next version of the CLF Internet standards
•	Introduce a tool for departments and agencies to self-assess the quality of both official languages on their Web sites	

	Credibility	
Indicator(s)	On-line information is authoritative, up-to-date, accurate, and relevant
On-line information is trustworthy, and makes sense to citizens/clients	
Measurement Level and Technique	Measurement at the service level using a combination of citizen/client feedback and self-assessments	
Primary tool(s)/data source(s)	1)	Omnibus surveys – includes, e.g., Ipsos-Reid research, NFO Interactive surveys, TBS GOL Internet Research Panel
2)	Departmental reporting on GOL plans and progress – the reporting includes two types of data related to credibility: self assessment of processes to ensure that on-line information is credible, reliable, and makes sense to clients; and data from client satisfaction surveys (client feedback on the completeness, accuracy, and trustworthiness of on-line information)	
Summary of results achieved in 2003	Very positive – A high percentage of Internet users think that on-line information is up-to-date. They are more likely to think that they can find the most up-to-date information through the Internet than through any other single delivery channel. GOL services are using multiple strategies to ensure that on-line information is reliable and makes sense to users. While only a few of these services have directly assessed clients’ perceptions of the quality of information available on-line, direct client feedback to date has been positive.	
Raw data	Perceptions of federal Web sites
•	76% of Internet users think that federal information available on-line is up-to-date (GOL Internet Research Panel, 2004)
•	Internet users are more likely to think that they can find the up-to-date federal information through the Internet than through any other single delivery channel (Ipsos-Reid, 2003)
•	45% think that the most up-to-date information is available on-line, 27% through a visit to an office, 21% by telephone, and seven percent in paper documents
•	72% of Internet users agree or strongly agree that federal sites have the information they need (GOL Internet Research Panel, 2004)
•	According to another study, 48% of Internet users are somewhat (40%) to very (eight percent) satisfied with the amount of information available on-line about the federal government’s activities; satisfaction appears to increase with greater knowledge of federal Web sites, however – 62% of those with six or more visits are satisfied, but only 43% with between two and five visits and only 35% with one visit are satisfied (NFO Interactive, 2003)
•	The NFO Interactive finding that satisfaction with the amount of on-line information about the federal government’s activities increases as knowledge of federal Web sites increases supports the results from the GOL Internet Research Panel, as the GOL panel included a strong majority of participants who were recruited from the Canada Site and the three gateway sites, that is, participants with good knowledge of federal Web sites	
	Credibility (cont’d)	
	GOL services 
•	GOL departments and agencies report using various strategies tailored to the Internet to ensure that information is reliable and makes sense to clients; the strategy they most frequently mention is a Web publication process (33% of services providing data), including reviews to ensure comprehension by the intended audience and various standards
•	Other frequently mentioned strategies include review of content by senior managers and/or senior communications personnel (22% of services), client feedback (18% of services), and automated quality control techniques such as checking of links and data validation (12% of services)
•	22 of 130 GOL services (17%) report clients’ perceptions of the information available on-line; for a majority of these services, client feedback is positive – Internet users are satisfied with the completeness and accuracy of the information they find, and have a high degree of trust in it
•	Improvements requested include: help in filtering the information available on-line, greater clarity (plain language formats), and coverage of additional topics
Information management
•	Enhanced information management principles are in place as a result of the Management of Government Information Policy (MGI), which took effect May 1, 2003
•	A majority of departments and agencies have used or are currently using the IM Capacity Check, a self-assessment tool allowing departments and agencies to evaluate the state of their IM practices against the MGI Policy and best practices, and to enable management to develop plans for improvement
•	Departments and agencies have identified senior executives to oversee the implementation of the MGI Policy, and to develop a corporate culture that values information as a strategic business asset and a public trust
•	Most departments and agencies are collaborating through interdepartmental committees and working groups to develop new standards and guidelines for the sound management of information over its lifecycle, and to share best practices, in order to facilitate implementation of the MGI Policy	
Plans for improvement	•	Continued proactive development of Web content
•	Continued departmental implementation of the IM Capacity Check, and development of IM improvement strategies
•	Assessment of the adequacy of investment in IM	

	Critical Mass of Services	
Indicator(s)	The most commonly used federal information and transaction services are on-line by 2005, with an emphasis on increasing depth
An electronic delivery channel for all new federal services	
Measurement Level and Technique	Measurement at the service level using self-assessments	
Primary tool(s)/data source(s)	1)	Departmental reporting on GOL plans and progress – the reporting includes two types of data related to a critical mass of services: scores from a model to measure the maturity of on-line services; and corresponding deliverables by year
•	What is the service maturity model? The model is a way of conceptualising how on-line services become more sophisticated to Canadians, businesses, and international clients. Specifically, it is a three-stage, nine-point continuum that begins with the development of a simple Web presence, and concludes with services offering the capability either to pull and synthesise information from diverse sources or to complete transactions on-line; the underlying assumption is that the later stages build on earlier ones. (A complete description of the model can be found in Appendix #2 of this report.)
•	How does the model work? For each GOL service, departments and agencies self-assess both the current level of development – i.e., where it is on the nine-point continuum – and the target in future years. There is no requirement that services will reach “level 9” by 2005. Individual targets vary according to multiple factors. In some cases technological limitations make the most advanced levels of development impossible to achieve by 2005. Other factors may include: the need to build partnerships, security and privacy issues, legislative or policy barriers, and competing priorities. Regardless, targets align with general client expectations; they represent ambitious goals toward which services are progressing.	
Summary of results achieved in 2003	Very positive – Departments and agencies are on track to meet the key GOL goal of putting the most commonly used federal services on-line at the target level of functionality for each by the end of 2005. All GOL services are now at least partly on-line. Departments and agencies completed work on another 15 of these services in 2003, bringing the total to 45 of 130. More specifically, they made good progress this year in expanding and improving clients’ on-line access to information on a range of topics, e.g., science, culture, and foreign affairs. Departments and agencies also increased the number of GOL services – particularly those targeting businesses – that offer on-line transaction capability.	
	Critical Mass of Services (cont’d)	
Raw data	•	About 65% of GOL services are for Canadians, 26% for businesses, and three percent for international clients (about six percent are internal to government)
•	All GOL services are now on-line to some extent (“level 3” – basic information about the service is available on-line – or higher)
•	Departments and agencies completed work on another 15 GOL services in 2003; in total, 45 of 130 services (35%) have reached their targeted level of development 
•	27 of 63 information services (43%) are now complete; the most mature (“level 9”) are: IC’s “Corporate Name Search – NUANS”, NRCan.’s “GeoConnections”, and Stats. Can.’s “Communication and Dissemination”
•	18 of 67 transaction services (27%) are now complete; the most mature (“level 9”) are: Department of Foreign Affairs and International Trade’s (DFAIT’s) “Export/Import Controls System” and IC’s “Federal Incorporations”
•	Note, however, that the average level of development for GOL services completed in 2003 (5.1) is significantly less than those completed in 2002 (6.5); it appears that in a few of these cases, a conservative target was chosen to ensure that the 2005 deadline could be met; in others, the change appears to be the result of a better application of the service maturity model in 2003
•	By 2005, the average information service will be at a “level 6” – it will provide clients with a full range of interactive tools in order to access customised information on-line; the average transaction service will be at a “level 7” – it will provide clients with the ability to at least partially complete a binding transaction with the federal government on-line (e.g., submit an application on-line, but provide supporting documents through another channel)
•	The most mature GOL services are, and in the future will continue to be, those for businesses
•	Specifically, in 2003 the average service for businesses was at a “level 6”, for Canadians a “level 5”, and for international clients a “level 4”
•	In 2005, the average service for businesses will be at a “level 7”, for Canadians a “level 6”, and for international clients a “level 5”
Information services – current status
•	Overall, 13 of the 63 information services (21%) now allow clients to combine and synthesise information from different on-line sources – to “pull” the information they need at the appropriate level of detail (“levels 7-9”)
•	All but one of these services – Agriculture and Agri-Food Canada’s (AAFC’s) “Agri-food Trade” – are at the same level as 2002; they cover, e.g., business, consumer, and federal financial (e.g., budget) information	
	Critical Mass of Services (cont’d)	
	•	Another 43 information services (68%) offer clients extensive information holdings through the Internet, with interactive tools, navigational aids, client segmentation strategies, and plain language writing to facilitate access (“levels 4-6”)
•	The number of services at this stage has increased by almost 50% in the past year; they now cover, e.g., science, cultural, and foreign affairs information, as well as a modern electronic system for publishing federal legislation
•	Services that have improved noticeably in 2003 in terms of the functionality they offer Internet users (two or more levels on the service maturity model) include: Canadian Heritage’s (CH’s) “Cultural Participation and Engagement”, Canadian International Development Agency’s “Web-based Program and Policy Information”, Communication Canada’s (Com. Can.’s) “Government of Canada Publications”, National Defence’s (DND’s) “Critical Infrastructure Protection” and “Information Technology Security”, Indian and Northern Affairs Canada’s (INAC’s) “On‑line Access to Corporate Information”, and the Royal Canadian Mounted Police’s “Community, Contract, and Aboriginal Policing” and “National Police Services”
•	The final seven information services (11%) have put basic information sources (e.g., policies, relevant regulations, reports, frequently asked question pages, etc.) on-line (“levels 1-3”)
•	These services cover a mixture of agriculture, fisheries, and health-related topics; most will offer clients extensive information holdings through the Internet by 2005 (“levels 4-6”)
Transaction services – current status
•	Overall, 24 of 67 transaction services (36%) now allow clients to conduct their business with the federal government (e.g., complete and submit an application form, make a payment, etc.) through the Internet (“levels 7-9”)
•	The number of services at this stage has increased by more than 70% this year; the majority (12) are services for businesses
•	Services that have improved noticeably in 2003 in terms of the functionality they offer Internet users (two or more levels on the service maturity model) include: Atlantic Canada Opportunities Agency’s (ACOA’s) “Grants and Contributions Funding”, DFAIT’s “Export/Import Controls System”, IC’s “Measurement Canada”, and Justice Canada’s (Jus.’s) “Family Orders and Agreements Enforcement”
•	Another 30 transaction services (45%) now allow clients to communicate with the federal government, including allowing them to submit a range of information to the federal government, receive individualised responses to their submissions (where appropriate), and use interactive tools to find answers to their questions (“levels 4-6”)
•	The majority of these services (15) are for Canadians	
	Critical Mass of Services (cont’d)	
	•	The final 13 transaction services (19%) have put basic information sources (e.g., policies, relevant regulations, reports, frequently asked questions pages, etc.) on-line (“levels 1-3”)
•	The majority of these services (nine) are for Canadians; most will allow clients to conduct their business with the federal government through the Internet by 2005 (“levels 7-9”)	
Plans for improvement	•	Processes to ensure that any new federal services are developed for electronic delivery (e.g., Memoranda to Cabinet and Treasury Board submission formats, as well as oversight)	

	Service Transformation	
Indicator(s)	A fundamental rethinking of business processes; use of shared or common solutions where this makes sense
Collaboration with other departments, jurisdictions, the private sector, and NGOs in order to provide truly integrated service delivery from the perspective of citizens/clients	
Additional explanation of what is being measured	The GOL initiative is pursuing service transformation both at the “whole of government” and at the service level. 
All federal institutions are required to implement a “common look and feel” for their Web sites. This includes use of common symbols so that Canadians can easily distinguish between federal and non-federal programs and services, as well as a common layout and tools so that sites are easy to navigate. A series of gateways and portals have been created on the main Government of Canada Web site – the Canada Site. They are single points of access to related information and services organised according to the citizen’s/client’s perspective. Together, this work has transformed the electronic service face of the federal government.
The Secure Channel provides a common infrastructure as well as common authentication and security services. Common tools include Web metrics, a content management system for gateways and portals, common metadata training, and an on-line business transformation kit. These services and tools provide a foundation for more fundamental service transformation and integration.
In addition, the GOL initiative is encouraging departments and agencies not just to put services on-line, but also to transform the way they deliver them to clients. As the 2000 Framework for Government On-Line states:
“Business processes and practices within and between federal departments and agencies will be reviewed and revamped and where appropriate streamlined to combine service offerings, resulting in electronic client-centric services…There are various degrees of integration from simple hyper-linking of relevant information right through to multiple partners co-managing one end-to-end service. Departments and agencies will decide on the level of integration that best meets their clients’ requirements” (p. 6).	
Measurement Level and Technique	Measurement at the “whole of government” and at service level primarily using self-assessments	
	Service Transformation (cont’d)	
Primary tool(s)/data source(s)	1)	Departmental reporting on GOL plans and progress – the reporting includes three types of data related to service transformation: a model to measure the current level of transformation as well as what is expected by 2005; current and planned partnerships; and scores from the service maturity model (described in the critical mass of services section of this report)
•	What is the transformation model? The model is a way of conceptualising what transformation is, and how services transform over time in order to better meet client needs and expectations. Specifically, the transformation model is a four-stage continuum that begins with grouping of related information to facilitate access, and concludes with a complete and “seamless” service experience. (A complete description of the model can be found in Appendix #3 of this report.)
•	How does the model work, and what are the implications for the 2005 target? For each GOL service, departments and agencies self-assess both the current level of transformation – i.e., where it is on the four-stage continuum – and the future target. There is no requirement that services will reach the highest level by 2005. Transformation goals are set depending on benefits to clients, risks, departmental capacity, and available resources.
2)	Secure Channel roadmap, outlining plans for the provision and use of Secure Channel services	
Summary of results achieved in 2003	Mixed – While progress is being made in laying a foundation for fundamental service transformation, few GOL services will have achieved this objective by the end of 2005. Almost all services identify at least one portal on the Canada Site through which they are primarily accessible to individuals and businesses. There is also strong demand for the Secure Channel, a common tool enabling all departments and agencies to offer sophisticated on-line services to their clients. About one-tenth of GOL services are planning to achieve the highest levels of development by the end of 2005, as measured by the service maturity model (“levels 8-9” for information services and “level 9” for transaction services); a majority of them will involve federal and/or interjurisdictional partnerships. More than two-thirds of GOL services are currently doing some type of transformation work, but in most cases this is simply grouping related components together on‑line. About one-quarter of these services are targeting the top two levels on the transformation model by the end of 2005, integration of related offerings (e.g., agreements with other organisations to work together, easy movement across delivery channels, and joined-up back-office systems) and seamless service across organisations and channels, respectively. More fundamental service transformation efforts will need to extend beyond the GOL timeframe and investment in order to be realised.	
	Service Transformation (cont’d)	
Raw data	•	According to Accenture, Canada is the only country having "begun the move to the Service Transformation stage of e-government"; in 2003, the company rated Canada #1 in the world for the third year in a row, largely because of its work to date in transforming services (e-Government Leadership: Engaging the Customer, 2003)
“Whole of government” transformation
•	115 of 122 GOL services (94%) have identified at least one portal through which they are primarily accessible to individuals or businesses (seven services are internal to government and therefore are not linked to portals; Stats. Can.’s “Collections” provides data for multiple portals, and as such only identifies a link to the Canadians gateway)
•	In keeping with the “no wrong door” principle, these services are also accessible through departmental Web sites
•	Data on the use of gateways and portals by individuals and businesses can be found in the take-up section of this report
•	Currently, two applications – Canada Customs and Revenue Agency’s (CCRA’s) “Address Changes On-line” and Human Resources Development Canada’s (HRDC’s) “Records of Employment on the Web” – use the Secure Channel ePass for authentication, and six applications – Com. Can.’s “Government of Canada Publications” and IC’s “Bankruptcy and Insolvency”, “Canadian Intellectual Property Office”, “Competition Bureau”, “Federal Incorporations”, and “Spectrum” – use the Receiver General Buy Button for on-line credit card payments
•	29 departments and agencies (77 applications) indicate that they need the Common Registration Service in 2004, 22 departments (45 applications) that they need ePass, and 10 departments (18 applications) that they need the Receiver General Buy Button
•	Data on the use of ePass and the Receiver General Buy Button by individuals and businesses can be found in the security section of this report
Transformation of GOL services
•	90 of 130 GOL services (69%) currently involve federal and/or interjurisdictional partnerships
•	23 services involve federal partnerships, 18 interjurisdictional partnerships, and 49 both federal and interjurisdictional partnerships	
	Service Transformation (cont’d)	
	•	Five of 63 information services (eight percent) now allow clients to pull together information from different sources, resulting in a single information product with the appropriate amount of detail (“levels 8-9” on the service maturity model, discussed in the critical mass of services section of this report)
•	By 2005, seven information services (11%) will offer their clients this kind of functionality – AAFC’s “Agri-food Trade”, ACOA’s “Information Sharing and Exchange”, IC’s “Business Intelligence Products – Strategis”, “Consumer Services”, and “Corporate Name Search – NUANS”, NRCan.’s “Geographical Applications”, and Stat. Can.’s “Communications and Dissemination”
•	A majority of these services (four) will involve federal partnerships, and most (six) will involve partnerships with other jurisdictions
•	Two of 67 transaction services (three percent) have joined different elements of the transaction process on-line so that clients can fully complete a binding transaction in real time without leaving the Web site (“level 9” on the service maturity model)
•	By 2005, six transaction services (nine percent) will offer their clients this kind of functionality – CH’s “Parks Canada’s On-line Reservation and Ticketing System”, DFAIT’s “Export/Import Controls Systems”, HRDC’s “Employment Insurance Services for Individuals” and “Record of Employment on the Web”, IC’s “Federal Incorporations”, and INAC’s “Aboriginal Employment”
•	Half of these services will involve federal and half interjurisdictional partnerships
•	Based on filings, 88 of 130 GOL services (68%) are doing some transformation work; this will increase to 111 services (85%) by 2005
•	63 of these 88 services have grouped related components together (primarily at the federal level) to improve access
•	12 services have developed interoperability (both federally and across jurisdictions), and include some coordination across channels, to personalise service; 30 services will be at this level by 2005
•	11 services have integrated related offerings to provide clients with bundled solutions, including agreements with other organisations to work together, self-service and easy movement across delivery channels, and joined-up back-office systems; 25 services will be at this level by 2005
o	14 of the 25 are information services; three will be at a “level 8-9” on the service maturity model; a majority (nine) will involve interjurisdictional partnerships
o	11 are transaction services; none will be at a “level 9” on the service maturity model; most (nine) will involve interjurisdictional partnerships 	
	Service Transformation (cont’d)	
	•	Two services provide clients with complete and seamless service across organisations and channels – IC’s “Corporate Name Search – NUANS” and “Federal Incorporations”; 12 services will be at this level by 2005
o	Four of the 12 are information services; one will be at a “level 8-9” on the service maturity model; all will involve interjurisdictional partnerships
o	Eight are transaction services; three will be at a “level 9” on the service maturity model; a majority (five) will involve interjurisdictional partnerships 
•	62 of 130 GOL services (47%) report that they have simplified interactions with the federal government
•	41 of 130 GOL services (31%) report that they have bundled related components in ways that are not possible through traditional delivery channels (e.g., in-person, phone, mail)
Leading-edge technology and integrated service delivery pilots
•	In the early phases of the GOL initiative, several leading-edge technology and integrated service delivery pilots were funded
•	Interjurisdictional pilots included: AAFC’s “Farm Safety Nets Electronic Service Delivery”; Health Canada’s (HC’s) “Researcher Profile Database – CIHR” and “Provincial/Federal First Nations Telehealth Project”; IC’s “Business Start-up Assistant”, “Consumer Complaints Registration and Distribution”, and “On-line Joint Registration Pilot”; NRCan.’s “A Regional Lens on Canada’s Land and Resources”; and Public Works and Government Services Canada’s (PWGSC’s) “Electronic Delivery of Programs and Services”
•	GOL investments have also been made in business cases and proof of concept projects to “join up” services across federal departments and agencies, such as: the DFAIT-led e-CRM project, which tests the feasibility of using a single-client relationship management system to integrate services aimed at exporters or across jurisdictions; the PWGSC-led e-Contacts project, which tests the use of natural languages to interpret frequently asked questions and points callers or Internet users to the correct contact in federal, provincial/territorial, and municipal governments; and a business case for the IC-led BizPal, which would provide would-be entrepreneurs with a listing of all licences and permits at all levels of government needed to start a business	
Plans for improvement	•	Development of service visions for Canadians, businesses, and international clients that will include: strategic directions for service offerings, analysis of client preferences, implications for channel use, and opportunities for interjurisdictional cooperation	
	Service Transformation (cont’d)	
	•	Development of “catalytic projects” that drive integration across departments and agencies, jurisdictions, and channels, that change government’s relationship with clients, that reduce the number of steps needed to receive a service, improve transaction turnaround times, and save money, and that have both short-term deliverables and longer-term goals
•	Projects include: “National Routing for Vital Statistics”, “My Account for Individuals and Businesses”, “RésEau – The Canadian Water Connection”, “Third Party Privilege Management and Business Authentication”, “Common Business Authentication”, “Permits and Licenses (BizPal)”, and “Live, Learn, and Succeed in Canada”
•	Definition of what a longer-term, fundamental service transformation agenda would look like, including the principles that would guide this transformation effort, measurable and time-limited outcomes, and indicators that would track progress towards the achievement of these outcomes	

	Client Take-up	
Indicator(s)	Citizens/clients know what is available on-line
Take-up of ESD channel increases over time	
Measurement Level and Technique	Measurement at the service level using tracking of how clients engage the federal government, and at the “whole of government” level using citizen/client feedback 	
Primary tool(s)/data source(s)	1)	Departmental reporting on GOL plans and progress – the reporting includes four types of data related to take-up: the number of unique clients served per year; the distribution of these clients across service delivery channels according to how they complete transactions and access information products; the volume of interactions across channels per year (i.e., channel load); and the marketing plans for individual services
2)	Omnibus surveys – includes, e.g., Citizens First and Taking Care of Business studies, EKOS’ Information Highway studies, Ipsos-Reid research, NFO Interactive surveys, TBS GOL Internet Research Panel	
Summary of results achieved in 2003	Mixed – Client take-up of on-line federal services is increasing. Visits to the Canada Site and two of the three gateways increased noticeably in 2003. Some well-established GOL services now have more than 50% of their clients completing transactions on-line. Overall, the current take-up for transaction services (24%) is comparable to self-reported use of on-line banking in Canada (21-32% according to EKOS (2003)), a reasonable benchmark of what to expect. However, when CCRA’s “Individual Tax” is excluded, the average take-up rate drops considerably. Further, a majority of Canadians think that the federal government does not do a good job of marketing its on-line services, and are only moderately aware of what is available. Only a few GOL services have marketing plans for 2004 and/or take-up strategies. While data quality improved significantly in the past year, there are still gaps and inconsistencies in measurement methodologies that make it difficult to set realistic take-up targets.	
Raw data	Trends in overall Internet use
•	62% of Canadian households had at least one regular Internet user in 2002, up slightly from 60% in 2001 (Stats. Can., 2003)
•	More than 51% had at least one person who regularly used the Internet from home, up slightly from almost 49% in 2001
•	32% of Internet users do at least some of their banking on-line, up from 26% in 2002 (EKOS, 2003)
•	21% do all of their banking through the Internet
•	The number of experienced “net bankers” is increasingly common; 15% of all Canadians, and 48% of those who bank on-line, have at least two years of experience	

	Client Take-up (cont’d)	
	•	Education and income continue to influence Canadians’ use of the Internet (Stats. Can., 2003)
•	In 2002, 87% of households whose head possessed a university degree had at least one regular Internet user, compared with 66% whose head possessed a high school/college education, and 30% whose head had less than a high school education
•	In 2002, 88% of households earning $70,000 or more had at least one regular Internet user, compared with 75% of those earning $40,000-69,999, 51% of those earning $23,001-39,999, and 33% of those earning less than $23,000
•	Age also influences take-up of on-line services, although the percentage of Internet users in the 55-64 age group increased more than any other group in 2002 (Stats. Can., 2003)
•	The percentage of those 65 and older using the Internet also increased noticeably
Awareness of federal service delivery on-line
•	“Awareness of the on-line services that governments offer is modest, with the majority rating their awareness as ‘moderate’. Self-perceived awareness is higher among those people exposed to the content through their web surfing activities. There is a real opportunity for governments to increase the awareness of Internet content and services, especially among those unlikely to stumble upon them.” (NFO Interactive, 2003)
•	15% of Internet users rate themselves as very aware, 67% as moderately aware, and 15% as not at all aware (three percent say they do not know)
•	Only 25% of Canadians say that they know what the URL is for the main Government of Canada Web site (Ipsos-Reid, 2003)
•	Only 17% of this group (four percent of all Canadians) identified the correct URL, Canada.gc.ca (or gc.ca)
•	Only 24% of Internet users think that the federal government is doing a good (22%) to excellent (two percent) job in promoting on-line information and services (Ipsos-Reid, 2003)
•	However, 46% of Internet users recall hearing or seeing an advertisement for the Canada Site in the past few months (Ipsos-Reid, 2003)	

	Client Take-up (cont’d)	
	•	More generally, 57% of Canadians say that they are moderately (46%) to very (11%) familiar with the services that the federal government offers through the Internet, up from 43% in 2001 (EKOS, 2003)
•	Note that self-rated awareness in the EKOS survey is lower than in the NFO Interactive survey (mentioned above); one reason for this may be that the difference in the respondents – EKOS surveyed all Canadians while NFO Interactive surveyed Internet users, who as a group are likely to have more knowledge of what is available on-line than Canadians in general; another reason may be the difference between the questions – EKOS focused on federal services, while NFO Interactive asked about government services in general
•	15 of 30 GOL departments and agencies have plans to market their on-line services in 2004, but they only have plans for 29 of 130 services (22%)
•	Six of the 45 services (13%) that are complete have marketing plans for 2004
Use of gateways and portals
•	There were about 16 million visits to the Canada Site in 2003, a 21% increase since 2002; further, there were over 800,000 visits to the business gateway in 2003, a seven percent increase since 2002, and over 2.1 million visits to the non-Canadians gateway, a 63% increase since 2002
•	A visit is defined as “the sequence of interactions an Internet user has on a Web site without 30 minutes of inactivity”
•	There were about 1.2 million visits to the Canadians gateway in 2003, a 64% decrease since 2002; the reason is that links to portals on the Canadians gateway are now also on the homepage of the Canada Site on a rotating basis (as of April 2003); there has been a significant increase in the number of visits to these portals as visits to the Canadians gateway have declined
•	E.g., page views for the Public Safety portal increased by 144% from June‑September 2003 in comparison with the same period in 2002; and the number of page requests for the Persons With Disabilities portal in July 2003 were more than double the number of requests in January of the same year (that is, prior to the addition of portal links on the Canada Site)
•	The most frequently visited portals are Business Start-up, Jobs, and Going to Canada; the fastest growing in terms of traffic volume are Canada and the World, Financing, and Public Safety
Take-up of GOL services – channel volume
•	“The Internet/e-mail is now well established as the second most common channel for contact [with the federal government]; Web sites have become the key starting point for many Canadians” (EKOS, 2003)	

	Client Take-up (cont’d)	
	•	34% of Canadians say that their most recent contact with the federal government was through the Internet/e-mail, up from 27% in 2002 (EKOS, 2003)
•	41% say their most recent contact was by phone, about the same as in 2002 (42%), 17% by mail, down from 21%, seven percent in-person, down slightly from nine percent
•	Asked in a different way, 33% of Internet users say that their most recent contact with a government employee involved e-mail (NFO Interactive, 2003)
•	81% say that their most recent contact involved the phone, 24% in‑person, 10% mail, nine percent fax, six percent a comment form on a Web site, and three percent a public kiosk
•	In comparison with individuals’ overall use of service delivery channels, businesses use the mail, Internet, and fax more frequently, and they visit government offices less frequently (Institute for Citizen-Centred Service, 2004)
•	58% of businesses say they use the mail vs. 25% of individuals, 45% say they visit government Web sites vs. 30% of individuals, 35% say they use the fax vs. five percent of individuals, and 25% visit government offices vs. 48% of individuals
•	More businesses (74%) use multiple channels to access a government service than do individuals (50%) (Institute for Citizen-Centred Service, 2004)
•	A strong majority of Canadians say that the purpose of their most recent contact through the Internet was to obtain either general information (30%), or the answer to a specific question (28%) (GOL Internet Research Panel, 2004)
•	Overall, 51% of Canadians (66% of Internet users) have visited a federal Web site at least once in the past year, up from 42% (56% of Internet users) in 2001 (EKOS, 2003)
•	Similarly, a recent international benchmarking study found that 51% of Canadians have used on-line federal services/Web sites in the past year; overall, Canada currently ranks #6 in the world in terms of use of on-line federal services – the global average is 30% (TNS, 2003)
•	Based on filings, there were 449.8 million on-line interactions between clients and GOL departments and agencies in 2003, up from 291.4 million in 2002; this represents a significant growth both in absolute terms (a 54% increase), and relative to the other service delivery channels (an increase from 50% of the total volume of interactions in 2002 to 62% in 2003)
•	Information services are mostly responsible for this increase, with the volume of interactions growing by 88% (from 145 million interactions in 2002 to 280.6 in 2003)
o	In contrast, the volume of interactions for transaction services only grew by 16% (from about 146.5 million interactions to 169.2 million in 2003)	

	Client Take-up (cont’d)	
	•	The volume of interactions through IVR decreased from 79.4 million in 2002 to 70.1 million interactions in 2003, while the other channels remained fairly stable
Take-up of GOL services – where clients complete transactions
•	Based on filings, 24% of (a total 70.3 million) GOL clients (or their intermediaries) completed transactions on-line in the past year, up from 21% in 2002
•	27% of government clients (either from federal departments or agencies, or from other jurisdictions) completed transactions on-line, up from 17%, 25% of individual Canadians, up from 22%, 19% of businesses, up from 16%, and two percent of international clients completed transactions on‑line
•	One reason for the difference in take-up between Canadians and businesses is that large services for Canadians such as individual tax filing and job searches/applications have been available for several years, while comparable services for businesses such as tax filing have just been launched; however, departments and agencies expect businesses will significantly outpace Canadians in their use of on-line services over the next few years
•	Take-up of some types of transaction services is much higher, i.e., greater than 50%
•	E.g., HRDC’s “Employment Services”, 97-100%; CCRA’s “Commercial Customs”, 97%; Jus.’s “Family Orders and Agreements Enforcement Assistance”, 90%; PWGSC’s “Linguistic and Multicultural Services”, 90%; Public Service Commission’s (PSC’s) “Recruitment”, 87%; Canadian Firearms Centre’s (CFC’s) “Canadian Firearms Program” (registration applications), 65%; DFAIT’s “Trade Commissioner”, 57%; and INAC’s “Indian Registration/Certificate of Indian Status”, 50%
•	However, when CCRA’s “Individual Tax” is removed the percentage of clients completing transactions on-line decreases to 13% in 2002 and 18% in 2003 (in comparison, the overall average is 21% in 2002 and 24% in 2003)
•	When on-line job services are removed as well (DND, HRDC, INAC, PSC), this percentage further decreases to seven percent in 2002 and 12% in 2003	
	A note on data quality
•	There are still variations in the way that GOL departments and agencies track Internet use, and it is more difficult to track volume for the in-person delivery channel than for other channels, e.g., the telephone	

	Client Take-up (cont’d)	
	•	The percentage of clients completing transactions on-line for “mature” GOL services (“levels 7-9” using the model discussed in the critical mass of services section of this report) – to complete and submit an application form, make a payment, etc. – is 25%, i.e., only one percent higher than the average for all transaction services; this is counter-intuitive, as in most cases, if not all, services rating themselves lower than a “level 7” should not yet have actual transactional functionality; one possible explanation is that departments and agencies are sometimes using different definitions of what constitutes an on‑line transaction (e.g., use of calculators, as well as information about the transaction service as opposed to the transaction itself), thus inflating the overall take-up average
•	When CCRA’s “Individual Tax” is removed from this subset of mature services, the percentage of clients completing transactions on-line in 2003 decreases to 15%
•	In addition, data sets are less complete for 2000-02 in comparison with 2003, making it difficult to calculate future trends (and plan accordingly) with a high degree of reliability	
Plans for improvement	•	Establishment of common Web metrics to better understand clients’ use of gateways and portals, and to the identify the parts of the sites with which they are having difficulty navigating
•	Continued improvement in the quality of take-up information at the service level for all delivery channels
•	Development and implementation of more aggressive marketing, migration, and take-up strategies to encourage individuals and businesses to move to self-service channels where this is their preference
•	The marketing strategy will focus on three key themes: increased awareness (e.g., broad advertising campaigns in all media to inform citizens and businesses that the federal government has on-line services for them), maximum impact (e.g., coordination of marketing plans, including identification of common themes, audiences and opportunities), and targeted promotion of services
•	Enhancement of the “Navigating the Canada Site” e-learning tool to promote the use of transaction services available through this site	

	Client Satisfaction	
Indicator(s)	Increased satisfaction levels through high quality services that provide clients time, effort, and cost savings relative to other delivery channels
Robust and accurate citizen/client feedback drives the improvement and evolution of services	
Measurement Level and Technique	Measurement at the service and “whole of government” level primarily using citizen/client feedback	
Primary tool(s)/data source(s)	1)	Departmental reporting on GOL plans and progress – the reporting includes three types of data related to client satisfaction: data from client satisfaction surveys; self-assessments of client benefits; and information on service standards
2)	Omnibus surveys – includes, e.g., Citizens First and Taking Care of Business studies, TBS GOL Internet Research Panel	
Summary of results achieved in 2003	Mixed – While overall client satisfaction is relatively high, there are neither enough surveys of client satisfaction by departments and agencies nor sufficient use of satisfaction results to communicate with clients and develop service improvement strategies. Canadians’ overall satisfaction with federal services has increased from 60% in 1998 to 64% in 2003. Half of the GOL departments and agencies have recently done a client satisfaction survey for at least one of their services; the average score is four or “satisfied” (on a one to five scale ranging from “very dissatisfied” to “very satisfied”). A smaller number have done surveys specifically for the Internet delivery channel. Only a few GOL services have sought their clients’ perceptions of expected client benefits such as easier access and better information. Only one-third of departments and agencies have published the results of their satisfaction surveys, and a similar number have published service standards (two-thirds report having established service standards). 	
Raw data	•	According to Citizens First 3, Canadians’ satisfaction with 18 frequently used federal services is 64%, up from 60% in 1998 (Institute for Citizen-Centred Service, 2003)
•	Canada Post, CCRA tax services, HC information services, and HRDC employment services all contributed to this improvement, as did Canadians’ satisfaction with on-line service generally
•	Further, there is evidence that Canadians’ perceptions of service quality vary across delivery channels, and that on-line service delivery can raise overall satisfaction levels (Institute for Citizen-Centred Service, 2003)
•	Satisfaction with on-line government services is 68%, whereas satisfaction with services available by phone is only 56%
•	More than 80% who have used the on-line version of a government service would do so again
•	Over 50% of Canadians who want to receive government services through a different delivery channel would like to switch to the Internet or e-mail; this compares with only 21% who would like to switch to in-person service, 19% to the phone, and seven percent to mail	
	Client Satisfaction (cont’d)	
	•	Greater satisfaction with on-line service delivery is noteworthy, as 51% of Canadians (66% of Internet users) have visited a federal Web site at least once in the past year, up noticeably since 2001 (EKOS, 2003); more detailed results can be found in the take-up section of this report
•	According to the Taking Care of Business survey, businesses’ satisfaction with frequently used federal services is 62% (Institute for Citizen-Centred Service, 2004)
•	Businesses have a stronger allegiance to the telephone than do individuals (Institute for Citizen-Centred Service, 2004)
•	82% of businesses that have used the telephone to receive a government service would do so again (their strongest allegiance), while 73% of individuals would use it again
•	In contrast, 78% of businesses that have used the on-line version of a government service would do so again, while 81% of individuals would use it again (their strongest allegiance)
GOL services – satisfaction surveys
•	45 of 130 GOL services (35%), covering 15 of 30 departments and agencies, have surveyed their clients at least once in the past two years; the average satisfaction score is about four or ”satisfied” (where one is “very dissatisfied” and five is “very satisfied”)
•	14 of these 45 services, covering 10 of 30 departments and agencies, report publishing survey results on their Web sites
•	Only a few GOL services provide evidence in filings of using results from client surveys to drive service improvement (although note that this finding may be the result of the methodology used – i.e., the way the specific question was worded; it requires follow-up)
•	Examples of services that do use survey results in this way include: HC’s “Health Promotion”, IC’s “Federal Economic Development Initiative in Northern Ontario (FedNor)”, NRCan.’s “Geographical Applications”, Stats. Can.’s “Collection”, “Communications and Dissemination”, and “Stakeholder Relationships”
•	20 of 130 GOL services (15%) have surveyed client satisfaction with their Internet channel; the average satisfaction score for on-line delivery is about the same as the average overall score, about four or ”satisfied” (where one is “very dissatisfied” and five is “very satisfied”)
•	These results are consistent with findings from a recent omnibus survey, which reports that average satisfaction score is about four or ”satisfied" for federal Web site usage (where one is “very dissatisfied” and five is “very satisfied”) (GOL Internet Research Panel, 2004)
•	13 of these services have compared their Internet channel with other channels; most (nine) indicate that satisfaction with on-line delivery is the same as with other channels	
	Client Satisfaction (cont’d)	
	GOL services – service standards
•	56 of 130 GOL services (43%), covering 19 of 30 departments and agencies, report having service standards
•	32 of these 56 services also report having done a client survey in the past two years
•	26 services, covering nine of 30 departments and agencies, have published their standards
GOL services – benefits
•	Departments and agencies report that their GOL services offer a wide range of – and in most cases, multiple – benefits for clients
•	115 of 130 services (88%) are providing easier access
•	62 services (47%) are providing time savings
•	62 services (47%) are providing simplified interactions with the federal government
•	60 services (46%) are providing better information leading to increased education, skills, and professional development opportunities, and/or better information leading to increased business opportunities
•	41 services (31%) are bundling related components in ways that are not possible through traditional delivery channels (e.g., in-person, phone, mail)
•	24 services (18%) are providing cost savings
•	However, only 23 of the 115 GOL services listing easier access as a benefit (20%) report seeking clients’ perceptions of on-line access
•	Actual feedback is positive, with elements such as Web site clarity and ease of navigation typically rated as “good” to “very good”
•	Only 12 of the 60 GOL services listing better information as a benefit (20%) report seeking clients’ perceptions of the completeness, accuracy, and trustworthiness of the information available on-line
•	Actual feedback is mixed; some clients are very satisfied with the information currently on-line; others think that it is incomplete or does not fully meet their needs	
Plans for improvement	•	Establishment of baseline client satisfaction scores for all core Service Improvement departments and agencies, including feedback on expected client benefits such as easier access and better information (a letter was sent to Deputy Ministers in November 2003, requiring baseline scores to be set by March 31, 2004)
•	Implementation of follow-up client satisfaction surveys for those services that established baselines prior to 2003
•	Publication of client satisfaction results by service delivery channel	
	Client Satisfaction (cont’d)	
	•	More comprehensive implementation of the Common Measurements Tool across federal departments and agencies for client-facing services, as the Service Improvement Initiative requires
•	Establishment and publication of service standards reflecting client expectations, as the Service Improvement Initiative requires
•	Identification of Service Improvement Leads in all departments and agencies
•	Creation of a culture of continuous service improvement through the incorporation of Service Improvement Initiative objectives into reporting against the Management Accountability Framework (a tool to guide TBS assessments of departmental performance)
•	Implementation of benchmarking with similar departments and agencies using the database established at the Institute for Citizen-Centred Service	

	Security	
Indicator(s)	Use of the common infrastructure 
Adequate steps by departments and agencies to ensure that transactions are secure
Citizens/clients perceive that on-line services are secure	
Measurement Level and Technique	Measurement at the “whole of government” level using citizen/client feedback, and at the service level using self-assessments, client feedback, and tracking of how they engage the federal government	
Primary tool(s)/data source(s)	1)	Omnibus surveys – includes, e.g., EKOS’ Information Highway studies
2)	Secure Channel roadmap, outlining plans for the provision and use of Secure Channel services
3)	Departmental reporting on GOL plans and progress – the reporting includes two types of data related to security: data on what type of IM/IT infrastructure is required by which date; and information on threat risk assessment, certification and accreditation, and business continuity planning
4)	Discussions among departmental security coordinators and departments and agencies with a lead role in this area	
Summary of results achieved in 2003	Mostly positive – Canadians have more confidence in governments than either the banks or the private sector in terms of offering safe on-line services. However, they have significantly more comfort in applying on-line for a program or service than for making an e-payment. The current take-up of on-line federal services also provides evidence of Canadians’ increasing confidence – the proportion of clients who completed transactions on-line increased from 21% in 2002 to 24% in 2003. In general, departments and agencies are meeting the security expectations of their clients. A reliable “whole of government” incident handling process is in place to respond to external threats. The Secure Channel now offers authentication and e-payment services, both of which Canadians and businesses are using in increasing numbers. Secure Channel is developing and implementing services for which there is strong departmental demand, enabling the federal government to conduct secure on-line transactions with clients. Departments and agencies are at various stages in implementing the Government Security Policy, which requires threat and risk assessments and business continuity plans for federal services.	
Raw data	Perceptions of on-line service delivery
•	Canadians’ awareness of security related technologies is modest, but increasing (EKOS, 2003)
•	Canadians have more confidence in governments than banks or the private sector in terms of offering safe on-line services (EKOS, 2003)
•	62% agree (20% disagree) that governments would not offer the choice of doing things as filing taxes through the Internet unless it was safe to do so, up from 59% in 2002
•	In contrast, 59% agree (24% disagree) that banks would not offer the choice of banking on-line unless it was safe to do so, and 38% agree (39% disagree) that companies would not offer the choice of doing things such as buying products through the Internet unless it was safe to do so	
	Security (cont’d)	
	•	73% of Canadians have a moderate (58%) to high (15%) level of comfort with applying on-line for a federal program or service, up slightly from 71% in 2001; but only 37% have a moderate (27%) to high (10%) level of comfort with making an on-line payment to the federal government, up slightly from 35% in 2001 (EKOS, 2003)
•	Internet users have a higher level of comfort than Canadians in general – 60% of Internet users have a moderate (32%) to high (28%) level of comfort with providing their credit card number on-line in order to make a payment to the federal government
Security coordination
•	Incident handling – the ability to detect and respond to Internet viruses – is a major security issue; incidents such as Nachi, Blaster, and Sobig have the potential to make on-line services (i.e., their networks and/or systems) unavailable or untrustworthy if a reliable incident handling process is not in place
•	To ensure reliable incident handling, the federal government has developed, and is continually monitoring results from, a number of common/shared services, including:
•	Secure Channel Information Protection Centre (IPC)
•	PWGSC IPC
•	PSEP (formerly OCIPEP) Coordination Centre
•	Cyber Incident Coordination System triage unit
•	Alerts and advisories from PSEP and other government and non-government organisations
•	In addition to this, some departments and agencies have implemented their own incident handling processes
The common, secure infrastructure
•	85 of 130 GOL services (65%) require strong authentication and security services to enable two-way transactions; this represents an estimated 15 million transactions by 2009-10
•	To date, the federal government’s Secure Channel services include:
•	SCNet, which is advanced IP/VPNe network inter-connecting 130 departments and agencies, certified to “Protected A” and scaleable to “Protected B” (see the end of this section for definitions)
•	Security and authentication services, including ePass, which control access to on-line applications, protect privacy, and secure IT infrastructure from hackers
•	Payment services	
	Security (cont’d)	
	•	More than 115,000 ePasses have been issued since the launch of CCRA’s “Address Changes On-Line” (September 2002) and HRDC’s “Record of Employment on the Web” (May 2003), which are the first two services to use ePass for on-line registration and authentication
•	Almost 45,000 on-line credit card payments have been processed using the Secure Channel payment service since its launch (May 2003)
•	There is a strong demand for Secure Channel services: 29 departments and agencies (77 applications) indicate that they need the Common Registration Service in 2004, 22 departments (45 applications) that they need ePass, 18 departments (34 applications) that they need Digital Signature, and 18 departments (35 applications) that they need Encryption Services
Government Security Policy
•	Three of the most critical factors in IM/IT security are a Threat and Risk Assessment (TRA), a Business Continuity Plan (BCP), and Certification and Accreditation (C&A) of the service prior to implementation and operation (see the end of this section for explanations of these activities); these activities are mandatory under the Government Security Policy, which was updated in 2002, including for the Secure Channel
•	Departments and agencies are at various stages in implementing the Government Security Policy; more than 50% of GOL services reporting information have completed or are completing their TRA; less than 30% report that they have a BCP in place; GOL services have not consistently done C&A
•	There is some evidence suggesting that there is a lack of understanding of what is expected and, thus, that an awareness program and/or direct support may be required; training is either being set up, or is available for federal employees
•	Note that the Secure Channel has completed its TRA, BCP, and C&A
•	Related to the Government Security Policy, the Management of Information Technology Security Standard is currently available as a final draft; it includes such management controls as TRA, BCP, and C&A as well as technical and operational safeguards, which are defined in the context of the Protect-Detect-Respond-Recover cycle	
Plans for improvement	•	Incident handling across the federal government continues to evolve and improve; future plans include:
•	Enhancing the Secure Channel IPC 
•	Building interoperability and coordination capabilities among the Secure Channel IPC, the PWGSC IPC, and PSEP (formerly OCIPEP)
•	Developing common analysis services and solutions with an infrastructure overlay that will assist departments and agencies in establishing or enhancing their own detect and respond capabilities	
	Security (cont’d)	
	•	Developing an intrusion detection standard – if passed, Bill C14 will give departments and agencies the legal framework for intrusion detection, acknowledging the responsibility of the federal government both to securely manage their IM/IT assets and to respect individual privacy rights (note that while some guidance is currently available, the standard will not be finalised until Bill C14 is passed)
•	Departments and agencies will receive additional support through the completion of the following common/shared initiatives:
•	Detection, Analysis, and Response Infrastructure Model, led by PSEP, which will provide a comprehensive, secure infrastructure to support the detection, analysis, and response to cyber incidents within the federal government 
•	Threat and Vulnerability Analysis System, led by CSE, which will support the identification of and response to current and emerging safety and security cyber threats, and improve the monitoring and mitigation of risks to federal services
•	Cyber Incident Coordination System, led by PSEP, which supports the coordination of incident management across the federal government
•	Future development of the Secure Channel will include services that enable secure interoperability across departmental applications (horizontal delivery), such as privilege management, e-forms, and service exchange
•	The federal government is currently developing additional security standards and guidance in support of the Government Security Policy to assist business owners and security professionals in identifying security requirements and in implementing appropriate controls and safeguards; they include: Business Continuity Planning Program Standard, IT Security Zones Baseline Security Requirements (a joint CSE and TBS initiative), Security Risk Management Standard, Incident Management Standard, Intrusion Detection Standard, and the overarching Management of Information Technology Security Standard; these standards and guidance will provide departments and agencies with a common set of tools and solutions, and will be available by the end of 2004-05
•	Specific security monitoring, including maturity assessment, is planned; this includes a departmental IT Security Self-Assessment Program to assess compliance with federal security policies and standards; subsequent “whole of government” analysis of the data will provide an overview of security compliance, and will indicate the presence of appropriate security processes, controls, resources, and solutions
•	The federal government is also putting increased emphasis on security awareness and training for all employees, not just security professionals; the goal is to help create and foster a culture of security across the government, ensuring a better and more consistent understanding of the security requirements and expectations	

Terminology
“Protected A” – if its compromise could reasonably be expected to cause a low level of injury to private or non-national interests, for example, disclosure of an exact salary figure
“Protected B” – if its compromise could reasonably be expected to cause a medium level of injury to private or non-national interests; such information concerns an individual or an organisation, and is considered to be particularly sensitive
Security risk management
1.	Threat and Risk Assessment (TRA), including a Statement of Sensitivity (SOS) identifying and categorising assets in terms of their confidentiality, integrity, availability, and value based on the Operational Standard for the Identification and Categorisation of Assets
2.	Business Continuity Plan (BCP), including a Business Impact Assessment (BIA) documenting the relative priority/criticality of services and systems, and the maximum/minimum allowable down-time
3.	Certification and Accreditation (C&A) of systems and services, confirming that the security requirements have been fulfilled, and signifying that management has authorised the system or service for operation
4.	Vulnerability Assessments, which identify inherent vulnerabilities and recommend remedial action
5.	Active Defence, one based on the Protect-Detect-Respond-Recover cycle

	Privacy	
Indicator(s)	Adequate steps to protect personal information/individual privacy on-line
Citizens/clients perceive that on-line services offer good privacy protection	
Measurement Level and Technique	Measurement at the “whole of government” level using citizen/client feedback, at the departmental level using self-assessments, and at the service level using self- and third-party assessments	
Specific tool(s)/data source(s)	1)	Omnibus surveys – includes, e.g., EKOS’ Information Highway studies, Ipsos-Reid research
2)	Privacy Impact Assessments (PIAs) – federal institutions conduct PIAs for new or redesigned programs/services that raise privacy issues; they provide the results of these assessments to the Privacy Commissioner before implementation, and publish summaries of the results on their Web sites
3)	Departmental reporting on implementation of the Common Look and Feel Standards for the Internet (CLF) – federal institutions self-assessed their compliance with the seven CLF components (33 standards in total) using either all or a representative sample of their Web sites; the score for each component is an average of what all departments and agencies subject to CLF have reported; the focus, here, is reporting on consistent placement and use of important notices (especially privacy notices) (CLF component descriptions, related standards, and a self-assessment guide are available at: http://www.cio-dpi.gc.ca/clf-nsi/index_e.asp)	
Summary of results achieved in 2003	Very positive – A high percentage of Canadians think that the federal government has the appropriate safeguards in place to protect personal information submitted on-line; they have as much confidence in the federal government as the banks, and more confidence in the federal government than the private sector to protect this information. While many Canadians still have concerns about on-line transactions that require the exchange of personal information, the current take-up of federal services (from 21% in 2002 to 24% in 2003) provides evidence that these concerns are being addressed. Departments and agencies are ensuring that protection of privacy is a key consideration from the initial design of on-line services and onwards; to accomplish this, they are conducting an increasing number of privacy impact assessments. While few Canadians understand the overall privacy stance of the federal government, including the extent to which personal information is being shared among departments and agencies, a majority believe that the privacy law is being followed.	
Raw data	General perceptions
•	34% of Canadians believe that federal departments and agencies have easy access to personal information, 25% that access requires a special request, 19% that access depends on personal consent, 18% that sharing with all other departments and agencies is prohibited (Ipsos-Reid, 2003)
•	66% of Canadians have a moderate (58%) to high (eight percent) level of confidence that federal departments and agencies will follow privacy laws regarding the use of personal information, a decrease from 73% in 2001 (EKOS, 2003)	
	Privacy (cont’d)	
	•	70% of Canadians do not mind governments using personal information as long as they know about it and can stop it, up from 66% in 2001 (EKOS, 2003)
•	In contrast, 48% do not mind banks, and 46% do not mind companies (down slightly from 50% in 2001), using personal information as long as they know about it and can stop it
•	49% of Canadians are comfortable with some sharing of personal information among federal departments and agencies if it means better and faster service, down from 54% in 2001 (EKOS, 2003)
•	60% are comfortable with some sharing of personal information among jurisdictions
Perceptions of on-line service delivery
•	54% of Canadians think they have enough information to know how the Internet may affect their privacy, up from 48% in 2001 (EKOS, 2003)
•	70% of Canadians are somewhat (30%), quite (17%), or extremely (23%) concerned about conducting transactions that require the exchange of confidential or personal information on-line (Ipsos-Reid, 2003)
•	About the same percentage of Internet users share this concern
•	However, 52% of Internet users have submitted personal information on-line, about the same as in 2001 (51%) (EKOS, 2003)
•	The information submitted includes: name (75%), e-mail address (74%), address (57%), home phone number (48%), credit card number (37%), income data (20%), and social insurance number (16%)
Perceptions of the federal government
•	Canadians have higher levels of trust in CCRA than any other major organisation to safeguard their personal information and not share it without permission (EKOS, 2003)
•	Internet users have as much confidence in the federal government as the banks, and more confidence in the federal government than the private sector, in terms of being able to fully protect any personal information they submit on-line (EKOS, 2003)
•	74% are somewhat (59%) to extremely (15%) confident in the federal government’s ability to fully protect personal information submitted on-line
•	In contrast, 73% are somewhat (60%) to extremely (13%) confident in the banks’ ability, and 62% are somewhat (52%) to extremely (10%) confident in credit card companies’ ability, to fully protect personal information submitted on-line	
	Privacy (cont’d)	
	Privacy Impact Assessment (PIA) process
•	The PIA Policy came into effect on May 2, 2002; it outlines a process for determining the effects of program or service delivery on individual privacy, and ensures that the appropriate steps are taken to avoid, address, or mitigate any risks; this process also helps provide Canadians with the assurance that their personal information is protected whether it is transmitted through the Internet, in person, by telephone, or by mail
•	Departments and agencies are required to conduct a PIA for any new or redesigned program/service that raises privacy issues; specifically, they are required to conduct a PIA if there will be: an increased use of personal information, a broader target population, a shift from direct to indirect collection of personal information, use of personal information for purposes other than those for which it was collected, greater sharing of personal information, contracting out, the creation of a common personal identifier, or an anticipated negative public response 
•	To date, departments and agencies have submitted 41 PIAs and 16 preliminary PIAs to the Office of the Privacy Commissioner; the expectation is that these numbers will double in 2004-05
•	In his 2002-03 Annual Report to Parliament, the interim Privacy Commissioner noted that while the first PIAs submitted did not contain complete information, their quality is improving as departments and agencies gain more experience in using them
•	Impacts of conducting PIAs include: a reduction in the risk of non-compliance with privacy legislation and policies, assurance that privacy protection is built into programs and services at the outset, assistance to senior officials in making fully informed decisions, and promotion of an awareness and an understanding of privacy issues by the program and service delivery communities
Common Look and Feel Standards for the Internet (CLF) – Important Notices
•	For all departments and agencies subject to CLF, the average level of implementation of the four CLF standards supporting important notices (e.g., placement and content of privacy notices) is 93%, where each standard accounts for one-quarter of the total implementation level	
Plans for improvement	•	Independent assessments to ensure appropriate content in the privacy notices posted on federal Web sites, e.g., that they inform Internet users what type of information is being collected (automatically) and the purpose for which it will be used 	

	Efficiency	
Indicator(s)	Return on investment (ROI), cost avoidance, and operational efficiencies from on‑line service delivery	
Additional explanation of what is being measured	The data below are only indicative of cost savings, cost avoidance, and increased productivity to date. A full evaluation of mature GOL services would be necessary in order to provide conclusive results.
Many experts believe that it will take time to realise cost savings, as on-line service is a new delivery channel, and as departments and agencies have not closed existing delivery channels. It should be noted as well that the initial GOL objective was not to achieve a specific cost savings amount, but to improve the service quality and improve client satisfaction by making federal services available through the Internet.	
Measurement Level and Technique	Measurement at the service and “whole of government” level using self‑assessments	
Primary tool(s)/data source(s)	1)	Departmental reporting on GOL plans and progress – the reporting includes two types of data related to efficiency: productivity, cost savings, and cost avoidance; and impacts of on-line service delivery on client requests, physical points of presence (in-person, call, and mail centres), and human resources	
Summary of results achieved in 2003	Mixed – Almost half of GOL services are realising some combination of cost savings and cost avoidance; many are reinvesting savings to improve service delivery. As well, about two-thirds of these services are realising productivity benefits, allowing employees to focus on more complex and higher-value activities in response to client needs and requests. However, there is little evidence that GOL departments and agencies have set aggressive channel migration strategies (and take-up targets), where appropriate, in order to maximise savings from lower cost on-line transactions. Savings are mostly through less use of paper and postage resources, and typically are not significant in terms of amount. Further, there is a lack of detailed costing, departmental benefits, and channel impact data necessary for the effective management of service delivery across all channels. 	
Raw data	•	Overall, based on filings 91 of 130 GOL services (70%) report some type of cost savings, cost avoidance, and/or increased productivity benefits 
•	78% of transaction services report these benefits, as compared with 62% of information services
•	92% of mature transaction services (“levels 7-9” using the model discussed in the critical mass of services section) report these benefits
•	However, given that only 35% of GOL services have reached their target levels of functionality and that the current take-up of on-line services is 24%, benefits of on-line service delivery for departments and agencies will likely increase in the future
•	For further information, see the critical mass of services and take-up sections of this report	
	Efficiency (cont’d)	
	Cost savings and avoidance
•	Taken as a whole, 59 of 130 GOL services (45%) report cost savings and/or cost avoidance benefits
•	28 of these 59 services report cost savings, 12 cost avoidance, and 19 both savings and avoidance
•	16 services report a net dividend
•	Many services report that they either reinvest savings in order to improve service, or use them to offset higher technical and administrative costs
•	36 of 67 transaction services (54%) report cost savings and/or cost avoidance opportunities 
•	Based on filings, however, there is no correlation between realising cost savings (gross or net) and maturity of on-line transactional capability
•	The most frequently mentioned reason for cost savings is less use of paper and postage resources for mailing; also frequently mentioned is elimination of printing and storage of paper documents (e.g., reports); the magnitude of savings, when identified, is typically not significant
•	Only a few services also report cost savings from increased self-service and a reduction of physical points of presence, e.g., Citizenship and Immigration Canada’s (CIC’s) “Selection of Immigrants and Visitors to Canada” service, which has saved $24 million to date because of a reduction in the number of call centres, and PWGSC’s “Procurement and Disposal”, which saves $3 million per year because of a reduction in the number of warehouses
•	Correspondingly, only six of the 47 services reporting cost savings (13%) also report that the number of in-person, mail, and/or call centres decreased in 2003
Productivity
•	Almost half of GOL services (63), covering 25 of 30 departments and agencies (83%), report that client enquiries are, or are expected to become, more complex
•	At the same time, 81 of 130 GOL services (62%) report increased productivity benefits
•	About two-thirds of these services point to greater client self-service as the reason for increased productivity; about one-third specify that client self-service is allowing employees to devote their time to other, higher‑value activities
•	81% of the services reporting that client enquiries are becoming more complex also report increased productivity benefits 
•	Further, 27 GOL services (25% of those providing data) report that they have plans to redeploy staff by 2005; nine have already done so
•	20 of these 27 services (74%) are transactional	
	Efficiency (cont’d)	
	Channel management
•	Filings from departments and agencies suggest that a significant majority of their GOL services have had no impact thus far on other delivery channels, and predict that these services will not have any impact by 2005
•	Where GOL services do report impacts, the most noticeable trend is with regard to mail centres – 11 services (nine departments and agencies) report that they have decreased the number of mail centres, and in total 18 services (15 departments and agencies) report that they will do so by 2005
•	In addition, 10 services (nine departments and agencies) report that they will decrease the number of in-person centres by 2005 in response to decreasing demand for over-the-counter service
•	Almost as many GOL services report that the number of call centres will increase by 2005 (in total, 13), as will decrease (17)
•	All services reporting that the number of call (and in-person) centres will increase by 2005 also report that client enquiries will be more complex 
•	For about one-fifth of GOL services (representing about two-fifths of departments and agencies), there is no information about the impact that on‑line service delivery is having and/or will have on in-person, call, and/or mail centres – they do not predict the impact of the Internet on one or more of the other delivery channels in their filings
•	Consistent with this, aggregate volume data for 2001-03 indicate that while use of the Internet has increased significantly, use of the other service delivery channels (e.g., in-person, telephone agent) has either decreased only slightly or not at all
•	Some transaction services are predicting that the volume of interactions through the Internet will increase more than what calculated trends based on data from previous years would suggest; correspondingly, they are overestimating future decreases in in-person (and telephone agent) volume
•	As well, some information services are predicting that the volume of interactions through the Internet will not increase as much as what calculated trends based on data from previous years would suggest
•	These data may suggest that transaction services need to implement more aggressive channel migration strategies to encourage clients to use self-serve channels, and that information services may need to ensure that they can handle the increasing volume on the Internet channel (note however, that these findings may in part be the result of the smaller sample size used for the analysis – only services providing predictions for 2005 were used for the comparison)
•	More detailed results can be found in the take-up section of this report	
	Efficiency (cont’d)	
Plans for improvement	•	Improvement of department and agency assessments of channel impacts, benefits targets and capture strategies, and take-up target achievement information 
•	Development by departments and agencies of client migration and channel management strategies for services, using as a basis the service visions for Canadians, businesses, and international clients that are being developed	

	Innovation	
Indicator(s)	Use and demonstration of innovative Internet applications – establishment of Canada as a leader in the knowledge-based economy and society	
Additional explanation of what is being measured	The focus of this dimension of the GOL measurement regime is on information and communication technology (ICT) innovations and their economic impacts – on award-winning on-line services, “readiness” for e-commerce, and e-procurement/e-sales activities. At the same time, it is necessary to recognise that the GOL initiative has only an indirect impact on e-commerce and economic growth. These results are also influenced by global factors such as the political climate, and by country-specific factors such as employment levels, real disposable income, and educational attainment. 
(Note: broader innovation results can be found in Canada’s Performance, the annual report of the President of the Treasury Board to Parliament: http://www.tbs-sct.gc.ca/report/govrev/03/cp-rc_e.asp)	
Measurement Level and Technique	Measurement primarily at “whole of government” level using third-party assessments	
Primary tool(s)/data source(s)	1)	World Economic Forum (WEF) Networked Readiness Index – benchmarks the degree to which countries are capable of using technology to promote economic growth
2)	Economist Intelligence Unit (EIU) e-Readiness Index – benchmarks the degree to which countries are prepared to take advantage of, and can maximise, Internet-based commercial opportunities
3)	Stats. Can.’s Survey of Electronic Commerce and Technology – collects data on business/government Internet use and sales (with or without on-line payment)
4)	GTEC awards (http://www.gtecweek.com/) 
5)	Omnibus surveys – includes, e.g., EKOS’ Information Highway studies	
Summary of results achieved in 2003	Mixed – Only a minority of Canadians think that governments (federal, provincial, territorial) are developing and implementing innovative ICT. However, about one‑third of GOL departments and agencies win awards for their use of technology in service delivery each year. Internationally, Canada ranks in the top 10 in surveys benchmarking countries’ readiness to take advantage of Internet‑based commercial opportunities. There is also initial evidence that Canada is beginning to realise this potential for economic growth through the use of ICT.	
Raw data	•	There is a link between work involving ICT and innovation; according to a recent Stats. Can. survey, “more than three quarters of establishments in ICT service industries were innovative between 2001 and 2003, the highest proportion of all industries surveyed” (2004)
•	Stats. Can. defines innovation as “a new or significantly improved product offered to clients, or a new or significantly improved process”	
	Innovation (cont’d)	
	General perceptions
•	39% of Canadians think that governments are on the leading edge of new technology, down from 42% in 2001 (EKOS, 2003)
•	In contrast, 52% think that banks are on the leading edge in implementing new technology
2003 GTEC medallists (federal)
•	For innovative service delivery to citizens and businesses: GOLD, Employment Insurance Appli-Web, HRDC; SILVER, Electronic Data Reporting Security Infrastructure, Stats. Can.; BRONZE, Our Gift to Canada/e-Health Forms On-line, Veterans Affairs Canada
•	For enhancing government operations: GOLD, E-FILE On-line Plus, CCRA; SILVER, Permanent Resident Card, CIC; BRONZE, Port Facilities Information Register - Quebec Region, TC
•	For securing and managing information assets: GOLD, PRISM: A Technology Program for Managing Procedural Information at the House of Commons; SILVER: Trade Commissioner, DFAIT; BRONZE, Interactive Information Service, CCRA
•	For strategic information management: GOLD, Developing the Information Management Infrastructure to Support Modernising Services for Canadians, HRDC
•	For enabling e-government in Canada: GOLD, Authentication Services/e-Pass Canada, TBS and CCRA
•	For innovative cross-jurisdictional e-government projects: BRONZE, Canada-US Initiative for Marine Cargo, CCRA
•	For implementing medium to large IT projects: GOLD, Tactical Command, Control, and Communications System, DND
•	For managing the human dimensions of e-government: BRONZE, IT Branch Workplace Wellness Program, CCRA
Readiness to use ICT to drive e-commerce
•	Canada has one of the world’s 24 “core technology-innovating economies”, as measured by the number of US utility patents granted yearly (WEF Global Competitiveness Report 2002-03)
•	Within this group, Canada ranks #9	
	Innovation (cont’d)	
	•	Canada ranks #6 on the WEF Networked Readiness Index, up from #12 in 2002
•	Canada ranks #4 for the ICT environment component of this index – it is particularly strong in areas such as availability of scientists and engineers, overall infrastructure quality, and number of secure Internet servers
•	Canada ranks #5 for the ICT readiness component of this index – it is particularly strong in areas such as availability of mobile Internet access, availability of broadband access, firm-level innovation, and business Intranet sophistication
o	While Canada only ranks #20 in terms of government prioritisation of ICT, it ranks #9 in terms of actual on-line services
o	In comparison with other countries, Canadian individuals and government are more prepared to use ICT than are Canadian businesses
•	Canada ranks #10 for the ICT use component of this index – it is particularly strong in areas such as use of on-line payment systems, use of the Internet for coordination with customers and suppliers, presence of wireless business applications
o	However, Canada only ranks #48 in terms of the percentage of businesses using e-commerce
•	Canada ranks #10 on the EIU e-Readiness Index; its raw score in 2003 is virtually the same as in past years (8.20 in 2003, 8.23 in 2002 and 2001)
•	According to EIU analysis, “there appears to be a high degree of uniformity among top-rated countries: only 0.47 points (out of a possible 10) separates first place from tenth”
•	Canada ranks #1 on two dimensions of this index, business environment (e.g., strength of the economy, regulatory environment, competition policy, quality of infrastructure) and supporting e-services (e.g., availability of e‑business consulting and technical support services, industry-wide standards for platforms and programming languages)
•	Canada is weakest on the connectivity dimension of this index (e.g., mobile-phone penetration, level of competition in the telecom industry)
Impacts of ICT diffusion on e-commerce
•	32% of Canadians now do at least some banking on-line, up from 26% in 2002 (EKOS, 2003)
•	The number of experienced “net bankers” is increasingly common; 15% of all Canadians, and 48% of those who bank on-line, have at least two years of experience
•	“Net bankers” are doing more than half of all their banking on-line	
	Innovation (cont’d)	
	•	The proportion of private sector businesses selling goods and services through the Internet in 2002 was 7.5%, up marginally from 6.7% in 2001; however, the value of these sales increased significantly, from about $10.39 billion to $13.34 billion (Stats. Can., 2003)
•	E-commerce in 2002 accounted for only 0.6% of total private sector operating revenue
•	Large firms are responsible for most e-commerce activity
•	E-commerce remains volatile; in 2002, seven firms stopped selling through the Internet for every 10 that started
•	The value of business to business/government sales through the Internet in 2002 was about $9.65 billion, a 19.8% increase since 2001 (Stats. Can., 2003)
•	The proportion of businesses buying goods and services through the Internet increased from about 22% in 2001 to about 32% in 2002
•	The proportion of government organisations buying goods and services through the Internet increased from about 55% in 2001 to about 65% in 2002
•	The value of business to consumer sales through the Internet in 2002 was about $3.69 billion, a 58.5% increase since 2001 (Stats. Can., 2003)
•	Although still early, Canada’s ICT sectors are having a positive economic impact on aggregate productivity, e.g., about a one percent increase for the period 1996-2001 (OECD, ICT and Economic Growth, 2003)	

Appendix #1: Expected Outcomes
The current expected outcomes incorporate, and are broader in scope than, the original set articulated in 2000.
For convenience, the quality of the search capability on federal Web sites (first identified in 2000) is being tracked through client surveys; and access to integrated services as well as greater personalisation through integrated Web portals will be measured in greater depth and consistency through the accountability framework that these portals are developing. Note that individual examples of integrated services – like the interactive tool on the Doing Business with Canada portal for information about doing business with Canada, the diagnostic on the Exporting/Importing portal testing a company’s export readiness, the interactive tool on the Financial Benefits portal identifying the benefits for which Canadians might be eligible, and the diagnostic on the Seniors portal providing personalised information for Canadians experiencing either the death of a parent or a life care transition – as well as greater personalisation – such as the password-protected customised page of links to federal information and services as well as the personalised e-mail service available through the Canada Site – are also tracked and reported through the annual report on the GOL initiative that is tabled in Parliament.
For credibility, the quality of information principles (first identified in 2000) are being measured through a self-assessment tool, the IM Capacity Check. For critical mass of services, the ability to communicate electronically with clients and the level of functionality of on-line services are being measured through the service maturity model; the benchmark for electronic communication is “level 5” and beyond on the service maturity model, and for apply, file, enquire, and payments functionality it is “levels 6-9” of the model specifically for transaction services. For service transformation, partnerships with other jurisdictions are being tracked through departmental reporting (at the service and funded project level) on GOL progress. For security, progress towards a secure on-line transactional capability is captured through the broader Secure Channel roadmap, which also identifies some of the leading-edge technologies being developed in support of integration.
Current expected outcomes	Original expected outcomes (articulated in 2000)	
Convenience
•	A “no wrong door” approach
•	Federal Web sites are easily identifiable and easy to navigate
•	Relevant on-line information and services are put together in ways that make sense from a citizen/client perspective	
•	Access to integrated information and services through Web portals
•	Greater personalisation available through portals
•	An advanced search capability	
Accessibility
•	On-line information and services are accessible to persons with disabilities
•	On-line information and services are available in both official languages		
Credibility
•	On-line information is authoritative, up-to-date, accurate, and relevant
•	On-line information is trustworthy, and makes sense to citizens/clients	
•	Enhanced information management principles in place and in use 	
Critical mass of services
•	The most frequently used federal information and transaction services are on-line by 2005, with an emphasis on increasing depth
•	An electronic delivery channel for all new federal services	
•	Two-way e-mail communication (or an appropriate alternative when it is necessary to provide a secure exchange of information)
•	Key federal services available on-line, with apply, file, and enquire functionality
•	An electronic payments capability	
Service transformation
•	A rethinking of business processes; shared or common solutions where this makes sense
•	Collaboration with other departments, jurisdictions, the private sector, NGOs in order to provide truly integrated service delivery from the perspective of citizens/clients	
•	Interjurisdictional ESD pilots
•	Testing of leading-edge technologies to improve and integrate service delivery	
Take-up
•	Citizens/clients know what is available on‑line
•	Take-up of the ESD channel increases over time		
Client satisfaction
•	Increased satisfaction levels through high quality services that provide clients time, effort, and cost savings relative to other delivery channels
•	Robust and accurate citizen/client feedback drives improvement and evolution of services		
Security
•	Use of the common infrastructure 
•	Adequate steps to ensure that transactions are secure	
•	A secure transaction capability	
Privacy
•	Protection of personal information/individual privacy on-line		
Efficiency
•	Return on investment (ROI), cost avoidance opportunities and operational efficiencies from on-line service delivery		
Innovation
•	Use and demonstration of innovative Internet applications – establishment of Canada as a leader in the knowledge-based economy and society		

Appendix #2: Service Maturity Model
Maturity model – information services (developed by TBS)
Publish basic information (push)	
1.	Web presence: Summary information about the service is available on-line, e.g., contact numbers, office locations and hours of operation, a general description of the service, etc.
2.	Basic reference: Some basic documents are on-line. In many cases, however, clients still need to use off-line sources to access these documents.
Note:	“Basic documents” can include policies, relevant regulations and/or laws. They can also include reports, communications pamphlets, FAQs, etc., depending on the service in question. The key point to note is that these are documents that someone familiar with the service in question can easily and quickly identify, and that clients can easily obtain through off-line sources as well (e.g., offices, reference sections in libraries).
3.	Reasonably complete reference: A reasonably complete set of these basic documents are available on-line. In addition, processes are being considered/developed to ensure that on-line information remains well-managed, that it remains accurate and up-to-date (e.g., content management systems).
Note:	“Complete”, here, is understood only from the perspective of the service provider, not the client; and it focuses on documents that clients can easily obtain through off-line sources, not on the vast information holdings of services that could potentially be made available through new technologies.	

Customise information holdings (push/pull)	
4.	Deepening of holdings: Information is increasingly put on-line in response to client needs/requests; there is an emphasis on making information available to clients that is otherwise difficult to access, hidden, or not available through off-line sources. As well, processes are being established to ensure that on-line information remains accurate and up-to-date.
Note:	There is a clear distinction between the third and fourth stages of progress. In this fourth stage, there is a strong client focus, which guides decisions in making information available on-line. Services have also made clear progress in going beyond providing the “basic documents” of stage three. Depending on their clients, and on the type of information that they provide, services can adopt different strategies as they provide more information on‑line. They can treat information as a free public resource, and thus make it as comprehensive as possible; or they can see themselves as an authoritative source of information, and thus provide it more selectively. Note, however, that the “deepening of holdings” is an open-ended process; once begun, there is no end-point in putting information on-line. 	
5.	Segmentation: Clients are increasingly segmented into sub-groups, and information is increasingly tailored for them based on a broad range of user characteristics. Text is written in a plain language style that can be easily understood by the client group for which it is intended, different media are used to communicate information, and specialised portals are developed where appropriate. In addition, some on-line interaction between clients and service providers is possible. For example, clients can submit requests for reports, surveys, etc., and receive automatic confirmation-of-receipt notices. Other interactive tools are available on-line, but most specific questions are still answered through off-line channels.
Note:	“Segmentation” is more than just identifying clients using broad categories like individuals and businesses.
6.	Interactive tools: Clients can use a full range of interactive tools to receive customised information on-line. These tools can include, e.g., database queries (including cross-tabulation requests), calculators, mapping tools, intelligent response systems, real-time discussions, etc. Live and automatic navigational aids facilitate the use of on-line services, where appropriate. E‑mail management systems are in place to deal with the growing volume of correspondence, including the increasingly detailed questions as information becomes more readily available.
Note:	The above list of interactive tools is not intended as a checklist – it is neither exhaustive, nor may every example be appropriate for every service. The key criterion in determining whether services have “a full range of interactive tools” is what clients can reasonably expect to do on-line, given both their information needs and current technological possibilities.	
Provide client-defined access to information (pull)	
7.	Customised profiles: Individual clients can establish multiple profiles on-line and receive information based on these profiles. Subject to privacy guidelines, they can also receive unprompted suggestions, updates, and other information based on their user histories, traffic patterns, etc.
8.	Partial synthesis: Individual clients have partial capability to pull information from diverse on-line sources, and they have some tools to combine or synthesise it in order to create new information products.
9.	Reasonably complete synthesis: Individual clients have reasonably full capability to pull and synthesise information from diverse sources, which allows them to conduct detailed research on‑line. They can access more complete and reliable information on broad policy objectives or societal outcomes – e.g., safer communities, a cleaner environment, etc. This includes not just high level trends and other summary data, but also the reports behind these summaries, raw data, and other information on the specific programs that are thought to contribute to the achievement of these objectives. Not only are there multiple levels of information, clients are also able to choose the level that is appropriate for them, thus contributing to greater transparency, accountability, and engagement.	

Maturity model – transaction services (multiple versions in use in different jurisdictions and organisations, e.g. Accenture)
Publish (passive/passive)	
1.	Web presence: Summary information about the service is available on-line, e.g., contact numbers, office locations and hours of operation, a general description of the service, etc.
2.	Basic reference: Some basic documents relating to the service are on-line, e.g., policies, regulations, laws, most commonly used forms, etc. In many cases, however, clients still need to use off-line sources to access these documents.
3.	Reasonably complete reference: A reasonably complete set of basic documents is available on-line. In addition, processes are being considered or developed to ensure that on-line information remains well-managed.	
Interact (active/passive)	
4.	Partial communication: Some clients can communicate on-line with the service provider. For example, they can submit basic forms or documents. There is no confirmation of status, however, and clients receive the details of any outstanding issues through off-line sources.
5.	Full communication: All clients can communicate on-line with the service provider. In addition, some interactive tools are available on-line to respond to queries, but off-line sources are still an important source of information and guidance.
6.	Reasonably complete interaction: All clients can submit a full range of information relating to their transaction, and receive individualised responses to their submissions, where appropriate. There is a reasonably complete set of interactive tools to respond on-line to queries. Live and automatic navigational aids facilitate the use of on-line services, where appropriate. E-mail management systems are in place to deal with the growing volume of correspondence.	
Transact (active/active)	
7.	Partial completion: Clients can complete a transaction on-line; something binding takes place between them and the service provider. This transaction is either partial – e.g., clients can complete an application on-line, but need to use off-line sources to submit supporting documents or to process the application – or not available to everyone – e.g., the transaction is in the pilot stage, or is being rolled out in phases.
8.	Full completion: All clients can fully complete a binding transaction on-line. Confirmation of the transaction, however, does not happen until a later date, and may occur through off-line sources.
9.	Full completion in real time: All clients can fully complete a binding transaction in real time without leaving the service provider’s Web site. Confirmation of the transaction is instantaneous, and occurs either on-line or through an e-mail.	

Appendix #3: Transformation Model
(Developed based on service vision work for Canadians, businesses, and international clients)
1.	Grouping and rationalising of related information/transactions to facilitate access	
Components of this stage include:
•	Organisations: primarily working within and/or across departments and agencies
•	Channels: unique, with separate controls, structures, authorities
•	Offerings: one-size fits all
•	Enablers: use of horizontal delivery mechanisms at the federal level (e.g., gateways/portals), but still multiple points of presence that focus on the specific program and channel, separate departmental business processes and back-office systems, separate IM approaches	
2.	Development of interoperability across organisations and/or channels to personalise service	
Components of this stage include:
•	Organisations: working within and across departments and agencies and/or jurisdictions
•	Channels: interoperable, with “work-arounds” to move from one channel to the other
•	Offerings:, one-to-small group based on basic client segmentation; consistency across channels
•	Enablers: use of horizontal delivery mechanisms (e.g., Canada Business Service Centres), common service standards, interoperable back-office systems and business processes, removal of policy barriers	
3.	Integration in order to offer bundled solutions	
Components of this stage include:
•	Organisations: shared services within and across departments and agencies and/or jurisdictions (agreements to work together, etc.)
•	Channels: integrated, with seamless movement from one channel to another
•	Offerings: tend to be end-to-end; self-serve across different channels, tailored to specific challenges and requests
•	Enablers: rationalised points of presence, integrated cross-organisational business processes and back-office systems, common metadata and taxonomy, rationalisation of related policies	
4.	Provision of a complete and “seamless” service experience (differentiated offerings/common enablers)	
Components of this stage include:
•	Organisations: harmonisation across departments and agencies and/or jurisdictions; the client cannot distinguish between the various organisations involved (e.g., alignment of regulations, etc.)
•	Channels: used by more than one organisation, and collaboratively maintained and supported
•	Offerings: differentiated; based on the unique needs and circumstances of the clients
•	Enablers: shared points of presence that use integrated channels, transformational process management, shared services, shared knowledge base, enabling policies/legislation