ARCHIVED - Performance Measurement for the Government On-Line Initiative
This page has been archived.
Archived Content
Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.
Performance Measurement for
the Government On-Line Initiative
Table of Contents
Executive Summary
Through Canada's Government On-Line (GOL) initiative, the most commonly used federal information and transaction services will be available on-line by the end of 2005. The approach GOL has taken is citizen- and client-centred, as well as "whole of government"; the overall objective is to put federal services on-line in a manner that increases client satisfaction with services, while also improving the operational efficiency of service delivery.
This report measures progress towards expected outcomes for the GOL initiative, including:
- increased citizen and client focus through convenient on-line service delivery, accessibility for persons with disabilities and in both official languages, and the provision of credible and reliable information;
- better service quality through the availability of a critical mass of on-line services, robust take-up of and satisfaction with on-line services, and the transformation (or rethinking) of services from the citizen's and client's perspective;
- increased trust and confidence in on-line service delivery through good security and privacy safeguards; and
- positive economic impacts through greater efficiency in service delivery and innovative services.
Data sources to assess performance across these eleven dimensions include: citizen and client feedback from a range of surveys, departmental self-assessments of GOL progress, and third-party analysis and benchmarking.
In 2003, the GOL initiative remained on track in terms of putting the 130 most commonly used federal services on-line, and developing their target levels of functionality. The achievement of these target levels of functionality will be the key measure of success for the Government of Canada's commitment to put the most commonly used federal services on-line by the end of 2005.
The Internet is now the second most frequently used method of contacting the federal government. A high percentage of Canadians think that on-line federal services are easy to access and use; a high percentage report that they could find the information they were seeking and that it was up-to-date.
Canadians have more confidence in governments than either banks or the private sector in terms of offering safe on-line services. A reliable "whole of government" incident handling process is in place to respond to external electronic threats, and the Secure Channel – a comprehensive set of network infrastructure and operations services, security services, and enabling software tools – offers secure authentication and e-payment capabilities.
While many Canadians still have concerns about conducting on-line transactions that require the exchange of personal information, a high percentage think that the federal government has put appropriate privacy safeguards in place. Privacy Impact Assessments are conducted before services are put on-line to ensure that designs safeguard personal information – more than 50 services have conducted an assessment (or preliminary assessment) since this requirement came into place in 2002, a number that is expected to double over the next two years.
For the most part, departments and agencies have automated services, achieving modest improvements in client convenience and processing requirements, rather than completely rethinking or integrating them. Fundamental service transformation is more complex, risky, and costly; because of this, departments and agencies have been encouraged to determine the appropriate degree of transformation for each service. Progress is being made in laying the foundation for fundamental service transformation – for example, through the development of inter-departmental gateways and Web portals, a common secure infrastructure that supports interoperability, and integrated service delivery pilots. A key challenge in 2004 will be to define what a longer-term, fundamental transformation agenda would look like, including the principles that would guide this effort, measurable and time-limited goals, as well as indicators that would track progress towards their achievement.
Also in 2004, departments and agencies will continue their efforts to improve the navigation and format of Web sites, accessibility for persons with disabilities, and the quality of French language content. Canadians' awareness of on-line federal services remains modest – a targeted marketing strategy is needed, as well as channel migration and management strategies in order to encourage clients who are willing and able to move to self-service channels; greater take-up of on-line offerings will maximise departmental productivity, cost savings, and cost avoidance benefits. Departments and agencies also need to regularly conduct client satisfaction surveys, publish the results, and use them to establish service improvement strategies.
The full results from the GOL initiative will be realised over time. The federal government will continue to refine its performance measurement approach in order to strengthen both the reporting of progress as well as the use of this data to improve service delivery to Canadians.
1) Introduction
Through Canada's Government On-Line (GOL) initiative, the most commonly used federal information and transaction services will be available on-line by the end of 2005.
The approach the GOL initiative has taken is both citizen-/client-centred and "whole of government"; its overall objective is to put federal services on-line in a manner that increases satisfaction with services, while also improving the operational efficiency of service delivery. The following report summarises the specific outcomes that are expected, the results achieved to date, and the actions needed to improve performance and strengthen the measurement regime.
The emphasis on outcomes is a reflection of the fact that four years have passed since the 1999 Speech from the Throne announcing the GOL initiative. The most commonly used federal services are now at least partially on-line. Because of this, it is appropriate to focus on the impacts – or results – of these services, including the extent to which clients are using them, their perceptions of service delivery quality, the extent of interjurisdictional collaboration, and efficiencies resulting from electronic service delivery.
2) Performance Measurement Methodology
Measuring progress towards expected outcomes has been an important part of the GOL initiative since its inception. Initially, GOL departments and agencies were required to create an e-mail address that clients could use (one-way communication), to put information about key services on-line, to make key forms available electronically, and to establish a basic search capability on their Web sites. Departments and agencies reported at the end of 2000 that they had met these targets.
In addition, all federal institutions subject to Common Look and Feel Standards for the Internet (CLF) – a group that includes, but which is broader than, GOL departments and agencies – were required to implement these standards by the end of 2002, making their sites easy to recognise and navigate, ensuring equal access for persons with disabilities and in both official languages, and standardising the placement and content of important notices such as privacy and copyright. Using a self-assessment tool, in 2003 federal institutions reported on progress made in implementing CLF. The use of this assessment tool raised questions about its methodology, making it difficult to understand the current overall level of compliance with CLF. These questions will be resolved prior to the next reporting cycle. The aggregate results dealing with convenience, accessibility, and privacy are found in Section Four of this report.
A set of expected outcomes was also established for electronic service delivery (ESD) in 2000, emphasising greater functionality:
- key federal services available on-line, with apply, file, and enquire capability;
- an electronic payments capability;
- a secure transaction capability;
- access to integrated information and services through Web portals;
- an advanced search capability;
- two-way e-mail communication (or an appropriate alternative when it is necessary to provide a secure exchange of information);
- greater personalisation available through portals;
- enhanced information management principles in place and in use;
- interjurisdictional ESD pilots; and
- testing of leading-edge technologies to improve and integrate service delivery.
GOL departments and agencies were required to achieve these outcomes by the end of 2005.
The measurement regime
As experience with on-line service delivery was acquired from 2000 onward, a broader range of outcomes expected over a longer horizon was defined, focusing increasingly on the results for citizens/clients and the benefits for the federal government, such as client take-up of on-line services, higher levels of user satisfaction with service delivery, and cost savings or efficiencies as a result of ESD.
This approach was consistent with what other jurisdictions were learning, as well as what external performance measurement experts working in this field had recommended. For example, Gartner argued that
government organisations need a much wider range of metrics…Instead of measuring success based on rankings in Web service availability studies, governments should focus on metrics that demonstrate operational efficiency and value to constituents ("The Gartner Framework for e-Government Strategy Assessment", Strategy Analysis Report, 2002, p. 5).
In 2002, therefore, the Treasury Board of Canada Secretariat (TBS) collaborated with Accenture, a multinational consulting firm with an extensive e-government practice, and a group of GOL departments and agencies to develop a more results-based measurement regime. The first step was a review of existing literature on e-government performance measurement. Overall, the findings were that this was a relatively new area, and that little work had yet been done. Nevertheless, the review did identify two good examples from which Canada could learn: Singapore and Ontario. Singapore uses an on-line self-assessment questionnaire to measure e-government progress in multiple areas. Ontario includes client satisfaction as one of the key expected outcomes of e-government, which (like the Government of Canada) it assesses directly with clients using a standardised methodology, the Common Measurements Tool (CMT). The environmental scan also incorporated learning from other jurisdictions, including a recent report from the National Audit Office in the United Kingdom, which argued for more emphasis on monitoring the take-up and marketing of on-line services (Better Public Services Through e-Government, 2002).
The second step was to establish the principles – a foundation – from which the GOL initiative could evolve its measurement regime:
- develop a broad perspective of success;
- recognise that ESD is one of several delivery channels that departments and agencies use;
- recognise that developing on-line services is only an intermediary step;
- ensure that the approach taken is flexible; and
- make reporting easy to implement, and useful for departments and agencies.
Perhaps the most important of these principles is developing a broad perspective of success. This is because the number of services available on-line does not matter if clients have difficulties in accessing them; and similarly, the sophistication of these services is not relevant if clients do not like and do not use them. Another important principle is recognising that the development of on-line services is only an intermediary step towards a larger goal. This is to say, GOL is not simply about meeting a deadline; rather, it is about enabling departments and agencies to better achieve their mandated objectives. Further, it is necessary to ensure that the approach taken is flexible. Measurement strategies need to recognise, and account for, the significant differences that exist among departments and agencies – those offering primarily information services versus those offering large transaction services, and those providing benefits versus those ensuring regulatory compliance.
With this foundation, a set of shared expected outcomes was established:
- increased citizen/client focus through convenient on-line service delivery, accessibility for persons with disabilities and in both official languages, and the provision of credible and reliable information;
- better service quality through a critical mass of on-line services and transformation (or rethinking) of services from the citizen's/client's perspective, as well as greater take-up of, and satisfaction with, these services;
- increased trust and confidence in on-line service delivery through good security and privacy safeguards; and
- positive economic impacts through greater efficiency in service delivery and innovative services.
These expected outcomes are consistent with the "commitments to Canadians" made in the first overview report on the GOL initiative. They also incorporate the original set of expected outcomes articulated in 2000 (see Appendix #1).
The final step was to determine the techniques that would be used to assess progress towards these expected outcomes, and the levels at which they would be assessed. In terms of technique, the GOL measurement regime combines self-assessment data from departments and agencies with citizen/client feedback. Self-assessments allow the individuals who are closest to, and thus have the clearest understanding of, the on-line services being developed to measure their progress. They are also relatively inexpensive and quick to do. Even so, departments and agencies sometimes have a different understanding than their clients of, for example, the benefits of a particular on-line service. There are also aspects of e-government that they are not in a position to accurately assess, such as confidence in transacting securely on-line. For these reasons, it is necessary to balance departmental self-assessments with data from citizen/client surveys as well as focus groups. This use of multiple types of data also ensures a full and robust gauge of results achieved to date.
In terms of level of measurement, the focus is on services, departments and agencies, and/or the "whole of government", as appropriate. For example, client take-up is always measured at the service level; implementation of CLF is a departmental way of improving recognition and accessibility of Web sites, and thus should be measured at the departmental level; integrated Web portals are a "whole of government" tool to improve convenience, and thus should be measured at that level.
Data sources
A key element of successful performance measurement is a data collection process that is both easy to implement, and useful for departments and agencies. To the extent possible, therefore, the GOL measurement regime relies on existing data sources. The following describes these data sources as well as some of the key assessment tools that are being used to assess the progress of the GOL initiative.
Departmental reporting on GOL plans and progress– Departments and agencies provide annual reports on progress to date in developing on-line services and integrated Web sites and portals. For example, in terms of on-line services they provide information on project milestones and deliverables, take-up, client benefits and satisfaction ratings, partnerships, departmental benefits, infrastructure needs, communications and marketing plans, as well as the implementation of select policies. Standardised tools to help departments and agencies with their assessments include:
- a three-stage, nine-point model measuring the increasing sophistication both of transaction services – from publish, to interact, to transact – and information services – from publish (basic information is available on-line), to customise (holdings are increasingly interactive), to provide client-defined access (users can increasingly manipulate/synthesise information from different sources);
- a four-stage model measuring the degree of service transformation – from grouping related components together in order to facilitate access, to interoperability and some coordination across channels in order to personalise service, to integration of related offerings in order to offer bundled solutions (such as agreements with other organisations to work together, easy movement across delivery channels, and joined-up back-office systems), to a complete and seamless service across organisations and channels; and
- the CMT, an interjurisdictional tool adopted by departments and agencies to measure client satisfaction across service delivery channels and across organisations.
Institutional reporting on CLF implementation – The target date for implementation of CLF was December 31, 2002. Federal institutions used a self-assessment tool in 2003 to review the progress made in implementing CLF, to identify any remaining barriers, and to set a target date for achieving full implementation (if applicable). Their summary reporting on certain CLF components – navigation and format, accessibility, official languages, and important notices – is particularly relevant for this performance measurement exercise. In terms of the methodology or approach, each institution assessed either all or a representative sample of its Web sites against the various standards described in the CLF self-assessment guide. They assigned scores out of 100 for each CLF standard based on either their assessment of the progress thus far (e.g., towards having Web content in the plainest and simplest language possible), or the number of their Web pages/documents that were compliant (e.g., with HTML as the primary format). Institutional scores for each CLF component are a simple average of the standards belonging to it. The aggregate or "whole of government" scores reported in Section Four of this report are calculated in the same way; they represent the overall progress that federal government institutions subject to CLF have made to date, as opposed to the percentage of departments and agencies (or Web sites) that are fully compliant. (Note: because Web information is continuously being added and updated, it will be important to review the CLF progress of departments and agencies in the future.)
Citizen/client feedback – Departments and agencies that are involved in the development of gateways and integrated Web portals regularly evaluate their work using focus groups, surveys, and other citizen-/client-based research (in 2003, 26 such studies were done). In addition, TBS collects information on a range of issues – including ease of access and navigation, clarity of language, perceptions of privacy and security – through a GOL Internet Research Panel that it has established. TBS also monitors and, where appropriate, participates in public opinion research on ESD done by other government organisations and private sector consulting firms; research ranges from Statistics Canada's (Stats. Can.'s) Household Internet Use Survey to EKOS' Information Highway studies and the Institute for Citizen-Centred Service's Citizens First reports.
Other data sources – In addition to the above, the GOL measurement regime incorporates data from a number of other sources; for example,
- Web metrics;
- annual reports done by the Official Languages Commissioner and the Privacy Commissioner;
- external indices such as Stats. Can.'s Survey of Electronic Commerce and Technology, the Economist's E-Readiness Index, the World Economic Forum's Networked Readiness Index; and
- international comparisons of various e-government initiatives that are done by academic institutions like the Centre for Public Policy at Brown University, non-governmental organisations like the UN's Division for Public Economics and Public Administration, and private sector consulting firms like Accenture.
3) Logic Model
The purpose of a logic model is to identify the causal linkages between the activities of a policy, program, or initiative, the outputs, and the achievement of its outcomes – to highlight the steps that would demonstrate progress towards the final goal, and to help determine where to focus measurement efforts (TBS, "Guide for the Development of Results-based Management and Accountability Frameworks", 2001).
The following diagram and text describe the GOL logic model, showing the relationship between work on gateways and integrated Web portals, services, infrastructure, and policies, and the overall objective of increased satisfaction with federal services while at the same time improving the operational efficiency of the Government of Canada (although specific cost savings targets have not been set).
Diagram
Gateways and integrated Web portals
One of the most common barriers that clients encounter is not knowing which department or agency, or which level of government, is responsible for delivering which service. In response to this, the federal government, as part of the GOL initiative, has developed integrated Web sites and portals – single points of access to on-line information and services.
The short-term outcomes include increased convenience to clients of on-line federal information and services. Integrated Web portals are making services easier to find by organising information according to the client's, not the government's, perspective – that is, according to subject or audience – and by grouping related offerings together, beginning with the federal government but including services from other jurisdictions over time.
In the medium-term, integrated Web portals will help increase the take-up of on-line federal services by improving Canadians' awareness of the range of on-line services available to them, and help promote more fundamental transformation of service delivery from the citizen's/client's perspective. When clients can easily find the services they want through the Internet, they will be more likely to use this delivery channel rather than traditional ones, such as the phone or an in-person visit. As well, rationalising grouped information will help eliminate overlap, potential conflicts, and gaps among information and service offerings, as well as promote the rethinking and integration of business processes. This transformation work will require new governance models, and will extend beyond the GOL timeframe.
On-line services
In addition to improving access, the GOL initiative is developing an on-line option for 130 of the most commonly used federal information and transaction services – those services that touch the lives of the greatest number of Canadians.
The short-term outcomes are to increase the on-line availability of credible and relevant federal information, as well as a critical mass of services. Extensive information holdings increasingly are being made available through the Internet, either in support of on-line transactions or as services in their own right. As this happens, departments and agencies are upgrading processes to manage these holdings – to ensure that they are accurate and up-to-date – and are offering them in plain language formats. Putting the most commonly used federal services on-line is building and sustaining the momentum (the "critical mass") for e-government. These services are easy and convenient to use, thus encouraging clients to use the Internet to interact with the federal government on a regular basis.
The medium-term outcomes of a critical mass of appropriately marketed on-line services will include increasing the client take-up of these services, ensuring that all individuals and businesses that want to use the Internet channel do so. As clients become aware of the benefits that on-line services offer – including more complete information, time savings, cost savings, simplified interactions – and as they are able to conduct an increasing variety of transactions with the federal government through the Internet, they will use on-line services more frequently and for more types of transactions. Further, continued client demands for service improvement and lessons learned through the GOL initiative will help build capacity within the federal government for more fundamental transformation of service offerings and service delivery in the medium- to longer-term.
Fundamental service transformation – the complete rethinking of programs, business processes, and associated policies to achieve significant client benefits and federal savings – is not possible by the end of 2005. However, efforts to date in restructuring information, services, and Web sites according to client expectations provide a foundation and an impetus for more profound changes.
A common infrastructure
The GOL initiative is developing a "whole of government" Secure Channel to provide secure, responsive, and economical on-line access to the federal government, as well as common network, support, authentication, information management, electronic payment, and other services in response to department and agency needs.
The short-term outcomes are robust security and privacy protection for on-line federal services. The services currently provided by the Secure Channel guard against network intrusions, ensuring the integrity of Web sites and databases, facilitating the secure exchange of information both externally and internally, as well as enabling transactions through services such as secure electronic payments processing. The Secure Channel is one of the ways of providing better security than various alternatives, such as Secure Socket Layer (SSL), because it includes: end-to-end encryption of data, a common solution for the identification and authentication of clients, and the infrastructure required for digital signatures (which verify not only identity, but also the authenticity of documents).
The medium-term outcomes will include increasing the take-up of on-line services and providing key enablers for more fundamental service transformation. First, the security and privacy safeguards provided through the Secure Channel will build citizen trust and confidence in using on-line federal services. Moreover, a common infrastructure like the Secure Channel will provide key enablers for "joined-up" and horizontal government services – it includes a telecommunications network for all federal departments and agencies that can be extended to other jurisdictions/third party organisations, and in the future will offer common applications facilitating the integration of related services. Migration to a common infrastructure will also enable new services and functionality (for example, the secure on-line viewing by a client of all government accounts) at a lower cost than if departments and agencies were to build their own infrastructures.
In addition, the Secure Channel is an example of the innovation that the GOL initiative is achieving, as it is a dynamic public-private sector partnership developing and using world-class technology to improve government service delivery.
Leadership and direction
This component reflects the role of policies and strategies to direct and coordinate the federal presence on-line, and to guide the development of key aspects of on-line service delivery. The implementation of Common Look and Feel Standards for federal Web sites increases the convenience and accessibility of information and services: sites are easily recognisable through the use of Federal Identity Program identifiers, have common navigational features, and are designed to serve the needs of disabled persons as well as both official languages groups in Canada. Updated IT security standards provide direction for the security of on-line services. Privacy impact assessments ensure that services are only put on-line when they offer good privacy protection.
Skills development strategies support the development of a critical mass of services and will be necessary as a result of more fundamental transformation of service delivery. New skills and tools for both employees and managers will help enable the shift of service delivery to the Internet channel, but also to a multi-channel environment managed at the enterprise level. In addition, ESD and service improvement targets, enterprise architectures, and accountability frameworks will support citizen-/client-centred service and the sound management of IM/IT resources, as well as increase client satisfaction and the use of common services and applications, thus helping achieve the longer-term GOL objective of operational efficiency.
4) Government On-Line Results (2003)[1]
Summary of results achieved
Canada continues to do well in international benchmarks of progress in developing e-government. According to the 2003 Accenture study, e-Government Leadership: Engaging the Customer, Canada ranked #1 in the world for the third year in a row, largely because of its work to date in transforming services. Noted strengths of the Canadian approach include: the development and evolution of on-line services in response to citizen/client demand, a focus on a coordinated multi-channel service delivery strategy, and the implementation of horizontal solutions such as Common Look and Feel Standards for the Internet and the Common Measurements Tool.
In a follow-up study, Accenture suggested that there was a gap between Canada's current #1 ranking and the perceptions of its citizens (e-Government: The Citizen's View, 2003). For example, in a comparison of Internet users in six different countries – Australia, Britain, Canada, Singapore, Spain, and the United States – Canadians ranked #5 in terms of knowledge of the benefits of on-line service delivery, suggesting that more targeted marketing and communications strategies are needed.
Brown University ranked Canada #3 in the world in 2003 (Global e-Government). It points to: the high level of development of the Canada Site, gateways, and integrated Web portals; easily comprehensible reports, guides, and other publications; the personalisation and customisation options available on federal Web sites; and the wide availability of on-line privacy notices. Using a slightly different methodology, the United Nations (UN) ranked Canada #3 in the world in terms of e-participation and #6 in terms of the maturity of on-line services (Global e-Government Survey, 2003). For the UN, one of Canada's strongest e-government features is its focus on developing portals.
[1] On December 12, 2003, there was a reorganisation of departments and agencies, including of Canada Customs and Revenue Agency, Canadian Heritage, Communication Canada, Department of Foreign Affairs and International Trade, Environment Canada, Health Canada, Human Resources Development Canada, National Defence, Public Service Commission, and Public Works and Government Services Canada. References to specific departments and agencies are based on information they filed prior to that date. The next iteration of this performance measurement report will reflect the revised list of departments and agencies participating in the GOL initiative, and the specific services for which they are responsible.
These international benchmarks serve as a context for the results that the GOL initiative achieved in 2003, as assessed through its measurement regime.
A citizen/client focus – The GOL initiative has improved the convenience of on-line federal information and services. A strong majority of Canadians who use the Internet think that these services are easy to access and use; they rate the Internet channel as more convenient than the telephone or in-person visits. Partly because of this, the use of the Internet is increasing in popularity over time, and is now the second most frequently used method of contacting the federal government. There are still challenges, however, in effectively organising on-line information and services so that access is intuitive for all users, and in improving the performance of search engines on federal Web sites. Efforts to enhance on-line offerings in ways that are consistent with user feedback will continue. In addition, the roles and performance expectations for gateways and integrated Web portals will be clarified, and a performance measurement regime will be developed and implemented to periodically assess them.
For the most part, federal Web sites provide information and services in both official languages, although the quality of French language content is occasionally less than desirable or inconsistent with the English equivalent. The majority of these sites also provide descriptive or alternative text for elements such as sound, videos, and images. Doing so improves access for persons with disabilities who rely on Web readers or other assistive technologies. Departments and agencies need to continue to work to ensure that French language content is of a high quality, and that there is equitable access for everyone who wants to use the Internet to interact with the federal government.
A high percentage of Internet users think that on-line federal information is up-to-date; they are more likely to think that it is the most current information available. User satisfaction with the amount of information on federal sites varies, but it is clear from the results of surveys that satisfaction increases with greater knowledge of federal Web sites – implying that site layouts can be improved – and there is a healthy demand for more information to be put on-line.
Service quality – The 30 departments and agencies participating in the GOL initiative are on track to meet the key GOL objective of putting the 130 most commonly used federal services on-line by the end of 2005. Each of these services has a target for the level of functionality that will be available for Internet users by the end of 2005. By the end of 2003, 35% of GOL services had reached their targeted level of functionality. By the end of 2005, the average information service will be at a "level 6" – it will provide users with a full range of interactive tools and navigational aids in order to access and customise extensive information holdings on-line. The average transaction service will be at a "level 7" – it will provide clients with the ability to at least partially complete a binding transaction with the federal government on-line (for example, submit an application on-line but provide supporting documents through another service delivery channel).
Program managers are mostly automating existing services, and making modest improvements in client convenience and processing requirements. Few of them are planning more fundamental service transformation, as measured by a target "level 8-9" for information services (allowing clients to pull together information from different sources, resulting in a single information product with the appropriate level of detail) or "level 9" for transaction services (so clients can complete a transaction in real time without leaving the Web site), or as measured by the proportion of services which will provide complete and seamless service across all organisations and channels by the end of 2005.
GOL departments and agencies were free to determine the appropriate degree of service transformation for each service by the end of 2005. Fundamental service transformation means changing the form or structure of a service in order to achieve "order of magnitude" improvements in value to clients, and cost savings or other benefits to government. Compared with making incremental improvements, fundamental service transformation is much more difficult, risky, and costly. It also involves working across programs, departments and agencies, and jurisdictions to integrate services and move to common databases or back office systems. Doing this is much more complex and time consuming than improving single, siloed programs. It is for these reasons that the federal government has elected to take a cautious approach – to lay a foundation for fundamental service transformation through the development of a citizen-focused, enterprise-wide Web face of government as well as a common secure infrastructure that will support interoperability and the development of enterprise business and information architectures, and to learn from selective investments in strategic service transformation pilots and projects. Nonetheless, it is timely to establish a clear vision and outcomes for service transformation over the remaining period of the GOL initiative, as well as define objectives and outcomes for the following period.
Half of Canadians (66% of Internet users) report having visited a federal Web site at least once in the past year. However, reported awareness of the on-line services that the federal government offers is modest. Only half of GOL departments and agencies have plans to market on-line services in 2004; marketing plans only exist for 22% of services (and only for 13% of services that are complete in terms of having achieved their target level of functionality). A government-wide marketing strategy is needed, as is more effort by departments and agencies to increase the awareness of on-line services, particularly among Internet users.
Visits to gateways and integrated Web portals are increasing – the most frequently visited portals are Business Start-up, Jobs, and Going to Canada; the fastest growing in terms of traffic volume are Canada and the World, Financing, and Public Safety. The number of on-line interactions (queries, browsing for information, etc.) between clients and GOL departments and agencies grew 54% between 2002 and 2003; the growth was mainly due to information services. The proportion of clients completing transactions on-line is also increasing. In 2003, 24% of GOL clients completed transactions on-line, up from 21% in 2002. Individual income tax filing and job searches drove a large part of this take-up – without these services, the percentage of clients completing transactions on-line was 12% in 2003, up from seven percent in 2002. One of the challenges in preparing this report was the varying quality of data. More work is needed to establish better metrics to understand client use of federal Web sites, and to obtain more consistent data on client interactions and transactions by service. Not all departments and agencies set client take-up targets for on-line offerings, and more aggressive marketing and migration strategies are needed in order to encourage clients to move to self-service channels where this is their preference.
Client satisfaction with on-line offerings is moderately high, as can be seen from the increasing use of the Internet channel as well as public opinion and client surveys. There continues to be strong support among Canadians for the concept of organising Web sites according to the "no wrong door" principle, and the majority of users of federal sites report finding the information they were seeking. According to the 2003 Citizens First survey, Canadians' satisfaction with federal services is 64%. By channel, they are more satisfied with government services available through the Internet channel (68%) than through other channels such as the telephone (56%). Consistent with this, the average satisfaction score for GOL services that have measured satisfaction specifically with the Internet channel is about four, or "satisfied", on a five-point scale where one is "very dissatisfied" and five is "very satisfied".
Improving client satisfaction with on-line services is a key objective of the federal government. The specific target is to increase overall satisfaction with service delivery by 10% by the end of 2005 (from 60% to 66%). In addition, all departments and agencies with citizen-facing services must publish satisfaction targets and standards for service delivery (such as timeliness), conduct client satisfaction surveys periodically, and publish results achieved. Robust measurement regimes will drive service improvement. Overall, however, current performance measurement and communication of results are weak – only half of GOL departments and agencies have surveyed clients during the past two years, and one-third have posted these results on their Web sites; likewise, about two-thirds of departments and agencies have set service standards, but only one-third have posted these standards on their sites. Departments and agencies will need to establish and publish targets as well as performance reports.
Trust and confidence in on-line service delivery – In general, Canadians have confidence in the security and privacy of on-line federal services; their confidence in federal Internet offerings is on par with, or higher than, their confidence in on-line banking services. More effort is needed to communicate with Canadians, both Internet and non-Internet users, about the security and privacy safeguards that are in place for on-line transactions and interactions, particularly as more services benefit from Secure Channel services such as on-line client authentication and electronic payments. Robust security measures are needed to protect the confidentiality and integrity of the information collected by the federal government. Departments and agencies are at various stages in implementing the Government Security Policy, but are meeting the security expectations of their clients; this policy requires threat and risk assessments and mitigation strategies, business continuity plans, compliance assessment, and enhanced training for all employees to foster a better understanding of security requirements and expectations.
Positive economic impacts – Some of the early benefits identified for clients of on-line services are easier access and time savings. Departments and agencies are also seeking benefits such as cost avoidance, cost savings and/or increased productivity. Some cost savings and/or avoidance are being realised in almost half of GOL services, and savings are being reinvested in the on-line channel and better service delivery. About two-thirds of GOL services are realising some productivity benefits, allowing employees to shift their focus from routine and low-value transactions to more complex client requests and higher-value activities.
There is no evidence of significant savings from the shift to on-line services, however, and many would argue that this is because it will take time to realise savings, as the Internet is an additional channel and existing channels have not been closed. On the other hand, there is little evidence that GOL departments and agencies are setting explicit benefits targets for services. There is also a lack of reliable information on service delivery costs by channel, which, together with client migration strategies, is necessary for the effective management of service delivery across channels within departments and agencies, and at an enterprise level across the federal government. About two-fifths of GOL departments and agencies were unable in 2003 to predict the impact of the Internet channel on their in-person, call centres, and/or mail centres for at least one of their services. They will need to develop benefits capture, client migration, and channel management strategies that are aligned with the service visions for individuals, businesses, and international clients over the next year.
The detailed results follow. As mentioned above, the strategy has been to rely on existing data sources where possible, which vary in terms of scope and methodology. It will therefore be necessary to continue to work on strengthening data quality while ensuring that the collection process remains easy to implement from a departmental perspective. Further work is also required to determine whether, and where, additional performance benchmarks are appropriate.
|
|
Convenience |
|
Indicator(s) |
A "no wrong door" approach Federal Web sites are easily identifiable and easy to navigate Relevant on-line information and services are put together in ways that make sense from a citizen/client perspective |
|
Measurement Level and Technique |
Measurement primarily at the "whole of government" level using citizen/client feedback |
|
Primary tool(s)/data source(s) |
1) Omnibus surveys – includes, e.g., EKOS' Information Highway studies, Ipsos-Reid research, NFO Interactive surveys, TBS GOL Internet Research Panel 2) Departmental reporting on implementation of the Common Look and Feel Standards for the Internet (CLF) – federal institutions self-assessed their compliance with the seven CLF components (33 standards in total) using either all or a representative sample of their Web sites; the score for each component is an average of what all departments and agencies subject to CLF have reported; the focus, here, is on navigation and format (CLF component descriptions, related standards, and a self-assessment guide are available at: http://www.tbs-sct.gc.ca/clf-nsi/index-eng.asp ) 3) Meta-analysis of the citizen-/client-based research done by gateways and portals |
|
Summary of results achieved in 2003 |
Mostly positive – A high percentage of Canadians think that on-line federal information and services are easy to access. In addition, most report being able to find the information they are seeking. However, there continue to be challenges in effectively organising content in multiple ways so that Internet users can intuitively use federal Web sites regardless of the path they choose. As well, Internet users have mixed feedback about the performance of search engines on federal sites. Work is ongoing in each of these areas. There is also a lack of good comparable data on the progress that gateways and portals are making, e.g., the functionality they offer to clients, client take-up, and satisfaction scores. |
|
Raw data |
Federal Web sites – access/"no wrong door"
– 30% used a bookmark, 29% an Internet search engine, 28% the URL, six percent a link on non-federal sites, and three percent a link in an e-mail message
– 81% of Canadians who recently used the Internet to access a federal service think that it was easy to do so – In contrast, 66% of those who used the mail think that it was easy, 64% who went to an office, and 58% who used the telephone
– Satisfaction appears to increase with greater knowledge of the sites; 52% with between two and five visits in the past year are satisfied, but only 31% with one visit
Federal Web sites – use/navigation
– 83% of Internet users say that it is somewhat (49%) to very (34%) easy to find the FAQ page, and 77% say that it is somewhat (47%) to very (30%) easy to find the e-forms and services page, on the Canada Site (Ipsos-Reid, 2003)
– Of those who have used the top menu bar, 89% have used the search function, 50% the link to the Canada Site, 42% the "Contact Us" button, 24% the "Help" button, and 12% the language toggle
Common Look and Feel Standards for the Internet (CLF) – navigation and format
|
|
Plans for improvement |
|
Accessibility |
|
|
Indicator(s) |
On-line information and services are accessible to persons with disabilities On-line information and services are available in both official languages |
|
Measurement Level and Technique |
Measurement at the departmental level using self-assessments, and at the "whole of government" level using citizen/client feedback |
|
Primary tool(s)/data source(s) |
1) Departmental reporting on implementation of the Common Look and Feel Standards for the Internet (CLF) – federal institutions self-assessed their compliance with the seven CLF components (33 standards in total) using either all or a representative sample of their Web sites; the score for each component is an average of what all departments and agencies subject to CLF have reported; the focus, here, is on accessibility and official languages (CLF component descriptions, related standards, and a self-assessment guide are available at: http://www.tbs-sct.gc.ca/clf-nsi/index-eng.asp ) 2) Omnibus surveys – includes, e.g., Ipsos-Reid research 3) Meta-analysis of the citizen-/client-based research done by gateways and portals |
|
Summary of results achieved in 2003 |
Mixed – Almost all Canadians think that they can obtain service in either official language through the Internet at least most of the time; they think that they are more likely to receive this type of service through the Internet than through any other delivery channel. More generally, the federal government has made good progress in implementing CLF standards relating to accessibility and official languages, although work remains to be done in order to achieve 100% compliance with this policy. Other challenges include, e.g., providing TTY numbers on "Contact Us" Web pages and making all aspects of Web sites understandable through Web readers. One of the ways these challenges are being addressed is through the development of an Accessibility Domain Architecture, which offers guidance for departments and agencies in choosing technology that provides equitable access at the outset of projects. |
|
Raw data |
Access for persons with disabilities
– Remaining challenges include commercial off-the-shelf applications and legacy data that are partially inaccessible to those relying on assistive devices; departments and agencies are encouraged to provide the information necessary to obtain alternative formats such as Braille, and to continue their conversion efforts as well as the search for and sharing of solutions that address accessibility challenges
– Departments and agencies are encouraged to continuously review and test their Web sites
– Departments and agencies are encouraged in the December 2002 CLF best-practice document to add TTY numbers to contact information
– Features include: multiple font sizes for text, multiple colour schemes to improve contrast, links that are highlighted when an Internet user drags a mouse over them, and alternative site navigation using a keyboard rather than a mouse – Current information content includes: Transport Canada's (TC's) "Access to Travel" (information on accessible travel and transportation, including city maps in Braille), Industry Canada's (IC's) "Assistive Technology" (information on adaptive computer technologies), Natural Resources Canada's (NRCan.'s) "Mapping for the Visually Impaired", and links to federal disability information – Future plans include: links to provinces and territories and other related Web sites
– Personas include: Jane – no disabilities, Edith – a senior
citizen, Access in both official languages
– This compares with 91% of Anglophones and 87% of Francophones for regular mail services, 88% of Anglophones and 84% of Francophones for phone services, and 86% of Anglophones and 81% of Francophones for in-person services
– This is consistent with an earlier finding of the Official Languages Commissioner that "the quality of the content on government Web sites occasionally leaves something to be desired from one organisation to another" (French on the Internet, 2002)
– The Commissioner also highlighted some of key remaining challenges, e.g., the development of a policy for on-line publication of scientific and historical information in both official languages, and the publication of more specialised documents in French
– More detailed results can be found in the credibility section of this report |
|
Plans for improvement |
– Establish an Internet Advisory Committee Quality Assurance Working Group to revise the compliance measurement methodology – Increase small department and agency awareness of CLF requirements and best practices through presentations to the Small Agencies Administration Network and to committees such as the one for heads of IT for small departments and agencies – Encourage departments and agencies that have not yet established Centres of Expertise for the Internet to do so – Develop a communications strategy and products to promote and support CLF awareness and compliance – Hold a CLF and Centres of Expertise workshop to increase awareness and share information, tools, and best practices with the Web practitioner community – Review and propose changes for the next version of the CLF Internet standards
|
Credibility |
|
|
Indicator(s) |
On-line information is authoritative, up-to-date, accurate, and relevant On-line information is trustworthy, and makes sense to citizens/clients |
|
Measurement Level and Technique |
Measurement at the service level using a combination of citizen/client feedback and self-assessments |
|
Primary tool(s)/data source(s) |
1) Omnibus surveys – includes, e.g., Ipsos-Reid research, NFO Interactive surveys, TBS GOL Internet Research Panel 2) Departmental reporting on GOL plans and progress – the reporting includes two types of data related to credibility: self assessment of processes to ensure that on-line information is credible, reliable, and makes sense to clients; and data from client satisfaction surveys (client feedback on the completeness, accuracy, and trustworthiness of on-line information) |
|
Summary of results achieved in 2003 |
Very positive – A high percentage of Internet users think that on-line information is up-to-date. They are more likely to think that they can find the most up-to-date information through the Internet than through any other single delivery channel. GOL services are using multiple strategies to ensure that on-line information is reliable and makes sense to users. While only a few of these services have directly assessed clients' perceptions of the quality of information available on-line, direct client feedback to date has been positive. |
|
Raw data
|
Perceptions of federal Web sites
– 45% think that the most up-to-date information is available on-line, 27% through a visit to an office, 21% by telephone, and seven percent in paper documents
– The NFO Interactive finding that satisfaction with the amount of on-line information about the federal government's activities increases as knowledge of federal Web sites increases supports the results from the GOL Internet Research Panel, as the GOL panel included a strong majority of participants who were recruited from the Canada Site and the three gateway sites, that is, participants with good knowledge of federal Web sites |
|
GOL services
– Other frequently mentioned strategies include review of content by senior managers and/or senior communications personnel (22% of services), client feedback (18% of services), and automated quality control techniques such as checking of links and data validation (12% of services)
– Improvements requested include: help in filtering the information available on-line, greater clarity (plain language formats), and coverage of additional topics Information management
|
|
|
Plans for improvement |
|
Critical Mass of Services |
|
|
Indicator(s) |
The most commonly used federal information and transaction services are on-line by 2005, with an emphasis on increasing depth An electronic delivery channel for all new federal services |
|
Measurement Level and Technique |
Measurement at the service level using self-assessments |
|
Primary tool(s)/data source(s) |
1) Departmental reporting on GOL plans and progress – the reporting includes two types of data related to a critical mass of services: scores from a model to measure the maturity of on-line services; and corresponding deliverables by year – What is the service maturity model? The model is a way of conceptualising how on-line services become more sophisticated to Canadians, businesses, and international clients. Specifically, it is a three-stage, nine-point continuum that begins with the development of a simple Web presence, and concludes with services offering the capability either to pull and synthesise information from diverse sources or to complete transactions on-line; the underlying assumption is that the later stages build on earlier ones. (A complete description of the model can be found in Appendix #2 of this report.) – How does the model work? For each GOL service, departments and agencies self-assess both the current level of development – i.e., where it is on the nine-point continuum – and the target in future years. There is no requirement that services will reach "level 9" by 2005. Individual targets vary according to multiple factors. In some cases technological limitations make the most advanced levels of development impossible to achieve by 2005. Other factors may include: the need to build partnerships, security and privacy issues, legislative or policy barriers, and competing priorities. Regardless, targets align with general client expectations; they represent ambitious goals toward which services are progressing. |
|
Summary of results achieved in 2003 |
Very positive – Departments and agencies are on track to meet the key GOL goal of putting the most commonly used federal services on-line at the target level of functionality for each by the end of 2005. All GOL services are now at least partly on-line. Departments and agencies completed work on another 15 of these services in 2003, bringing the total to 45 of 130. More specifically, they made good progress this year in expanding and improving clients' on-line access to information on a range of topics, e.g., science, culture, and foreign affairs. Departments and agencies also increased the number of GOL services – particularly those targeting businesses – that offer on-line transaction capability. |
|
Raw data |
– 27 of 63 information services (43%) are now complete; the most mature ("level 9") are: IC's "Corporate Name Search – NUANS", NRCan.'s "GeoConnections", and Stats. Can.'s "Communication and Dissemination" – 18 of 67 transaction services (27%) are now complete; the most mature ("level 9") are: Department of Foreign Affairs and International Trade's (DFAIT's) "Export/Import Controls System" and IC's "Federal Incorporations" – Note, however, that the average level of development for GOL services completed in 2003 (5.1) is significantly less than those completed in 2002 (6.5); it appears that in a few of these cases, a conservative target was chosen to ensure that the 2005 deadline could be met; in others, the change appears to be the result of a better application of the service maturity model in 2003
– Specifically, in 2003 the average service for businesses was at a "level 6", for Canadians a "level 5", and for international clients a "level 4" – In 2005, the average service for businesses will be at a "level 7", for Canadians a "level 6", and for international clients a "level 5" Information services – current status
– All but one of these services – Agriculture and Agri-Food Canada's (AAFC's) "Agri-food Trade" – are at the same level as 2002; they cover, e.g., business, consumer, and federal financial (e.g., budget) information
– The number of services at this stage has increased by almost 50% in the past year; they now cover, e.g., science, cultural, and foreign affairs information, as well as a modern electronic system for publishing federal legislation – Services that have improved noticeably in 2003 in terms of the functionality they offer Internet users (two or more levels on the service maturity model) include: Canadian Heritage's (CH's) "Cultural Participation and Engagement", Canadian International Development Agency's "Web-based Program and Policy Information", Communication Canada's (Com. Can.'s) "Government of Canada Publications", National Defence's (DND's) "Critical Infrastructure Protection" and "Information Technology Security", Indian and Northern Affairs Canada's (INAC's) "On-line Access to Corporate Information", and the Royal Canadian Mounted Police's "Community, Contract, and Aboriginal Policing" and "National Police Services"
– These services cover a mixture of agriculture, fisheries, and health-related topics; most will offer clients extensive information holdings through the Internet by 2005 ("levels 4-6") Transaction services – current status
– The number of services at this stage has increased by more than 70% this year; the majority (12) are services for businesses – Services that have improved noticeably in 2003 in terms of the functionality they offer Internet users (two or more levels on the service maturity model) include: Atlantic Canada Opportunities Agency's (ACOA's) "Grants and Contributions Funding", DFAIT's "Export/Import Controls System", IC's "Measurement Canada", and Justice Canada's (Jus.'s) "Family Orders and Agreements Enforcement"
– The majority of these services (15) are for Canadians
– The majority of these services (nine) are for Canadians; most will allow clients to conduct their business with the federal government through the Internet by 2005 ("levels 7-9") |
|
Plans for improvement |
|
|
|
Service Transformation |
|
Indicator(s) |
A fundamental rethinking of business processes; use of shared or common solutions where this makes sense Collaboration with other departments, jurisdictions, the private sector, and NGOs in order to provide truly integrated service delivery from the perspective of citizens/clients |
|
Additional explanation of what is being measured |
The GOL initiative is pursuing service transformation both at the "whole of government" and at the service level. All federal institutions are required to implement a "common look and feel" for their Web sites. This includes use of common symbols so that Canadians can easily distinguish between federal and non-federal programs and services, as well as a common layout and tools so that sites are easy to navigate. A series of gateways and portals have been created on the main Government of Canada Web site – the Canada Site. They are single points of access to related information and services organised according to the citizen's/client's perspective. Together, this work has transformed the electronic service face of the federal government. The Secure Channel provides a common infrastructure as well as common authentication and security services. Common tools include Web metrics, a content management system for gateways and portals, common metadata training, and an on-line business transformation kit. These services and tools provide a foundation for more fundamental service transformation and integration. In addition, the GOL initiative is encouraging departments and agencies not just to put services on-line, but also to transform the way they deliver them to clients. As the 2000 Framework for Government On-Line states: "Business processes and practices within and between federal departments and agencies will be reviewed and revamped and where appropriate streamlined to combine service offerings, resulting in electronic client-centric services…There are various degrees of integration from simple hyper-linking of relevant information right through to multiple partners co-managing one end-to-end service. Departments and agencies will decide on the level of integration that best meets their clients' requirements" (p. 6). |
|
Measurement Level and Technique |
Measurement at the "whole of government" and at service level primarily using self-assessments |
|
Primary tool(s)/data source(s) |
1) Departmental reporting on GOL plans and progress – the reporting includes three types of data related to service transformation: a model to measure the current level of transformation as well as what is expected by 2005; current and planned partnerships; and scores from the service maturity model (described in the critical mass of services section of this report) – What is the transformation model? The model is a way of conceptualising what transformation is, and how services transform over time in order to better meet client needs and expectations. Specifically, the transformation model is a four-stage continuum that begins with grouping of related information to facilitate access, and concludes with a complete and "seamless" service experience. (A complete description of the model can be found in Appendix #3 of this report.) – How does the model work, and what are the implications for the 2005 target? For each GOL service, departments and agencies self-assess both the current level of transformation – i.e., where it is on the four-stage continuum – and the future target. There is no requirement that services will reach the highest level by 2005. Transformation goals are set depending on benefits to clients, risks, departmental capacity, and available resources. 2) Secure Channel roadmap, outlining plans for the provision and use of Secure Channel services |
|
Summary of results achieved in 2003 |
Mixed – While progress is being made in laying a foundation for fundamental service transformation, few GOL services will have achieved this objective by the end of 2005. Almost all services identify at least one portal on the Canada Site through which they are primarily accessible to individuals and businesses. There is also strong demand for the Secure Channel, a common tool enabling all departments and agencies to offer sophisticated on-line services to their clients. About one-tenth of GOL services are planning to achieve the highest levels of development by the end of 2005, as measured by the service maturity model ("levels 8-9" for information services and "level 9" for transaction services); a majority of them will involve federal and/or interjurisdictional partnerships. More than two-thirds of GOL services are currently doing some type of transformation work, but in most cases this is simply grouping related components together on-line. About one-quarter of these services are targeting the top two levels on the transformation model by the end of 2005, integration of related offerings (e.g., agreements with other organisations to work together, easy movement across delivery channels, and joined-up back-office systems) and seamless service across organisations and channels, respectively. More fundamental service transformation efforts will need to extend beyond the GOL timeframe and investment in order to be realised. |
|
Raw data |
"Whole of government" transformation
– In keeping with the "no wrong door" principle, these services are also accessible through departmental Web sites – Data on the use of gateways and portals by individuals and businesses can be found in the take-up section of this report
– 29 departments and agencies (77 applications) indicate that they need the Common Registration Service in 2004, 22 departments (45 applications) that they need ePass, and 10 departments (18 applications) that they need the Receiver General Buy Button – Data on the use of ePass and the Receiver General Buy Button by individuals and businesses can be found in the security section of this report Transformation of GOL services
– 23 services involve federal partnerships, 18 interjurisdictional partnerships, and 49 both federal and interjurisdictional partnerships
– By 2005, seven information services (11%) will offer their clients this kind of functionality – AAFC's "Agri-food Trade", ACOA's "Information Sharing and Exchange", IC's "Business Intelligence Products – Strategis", "Consumer Services", and "Corporate Name Search – NUANS", NRCan.'s "Geographical Applications", and Stat. Can.'s "Communications and Dissemination" – A majority of these services (four) will involve federal partnerships, and most (six) will involve partnerships with other jurisdictions
– By 2005, six transaction services (nine percent) will offer their clients this kind of functionality – CH's "Parks Canada's On-line Reservation and Ticketing System", DFAIT's "Export/Import Controls Systems", HRDC's "Employment Insurance Services for Individuals" and "Record of Employment on the Web", IC's "Federal Incorporations", and INAC's "Aboriginal Employment" – Half of these services will involve federal and half interjurisdictional partnerships
– 63 of these 88 services have grouped related components together (primarily at the federal level) to improve access – 12 services have developed interoperability (both federally and across jurisdictions), and include some coordination across channels, to personalise service; 30 services will be at this level by 2005 – 11 services have integrated related offerings to provide clients with bundled solutions, including agreements with other organisations to work together, self-service and easy movement across delivery channels, and joined-up back-office systems; 25 services will be at this level by 2005
– Two services provide clients with complete and seamless service across organisations and channels – IC's "Corporate Name Search – NUANS" and "Federal Incorporations"; 12 services will be at this level by 2005
Leading-edge technology and integrated service delivery pilots
– Interjurisdictional pilots included: AAFC's "Farm Safety Nets Electronic Service Delivery"; Health Canada's (HC's) "Researcher Profile Database – CIHR" and "Provincial/Federal First Nations Telehealth Project"; IC's "Business Start-up Assistant", "Consumer Complaints Registration and Distribution", and "On-line Joint Registration Pilot"; NRCan.'s "A Regional Lens on Canada's Land and Resources"; and Public Works and Government Services Canada's (PWGSC's) "Electronic Delivery of Programs and Services"
|
|
Plans for improvement
|
– Projects include: "National Routing for Vital Statistics", "My Account for Individuals and Businesses", "RésEau – The Canadian Water Connection", "Third Party Privilege Management and Business Authentication", "Common Business Authentication", "Permits and Licenses (BizPal)", and "Live, Learn, and Succeed in Canada"
|
Client Take-up |
|
|
Indicator(s) |
Citizens/clients know what is available on-line Take-up of ESD channel increases over time |
|
Measurement Level and Technique |
Measurement at the service level using tracking of how clients engage the federal government, and at the "whole of government" level using citizen/client feedback |
|
Primary tool(s)/data source(s) |
1) Departmental reporting on GOL plans and progress – the reporting includes four types of data related to take-up: the number of unique clients served per year; the distribution of these clients across service delivery channels according to how they complete transactions and access information products; the volume of interactions across channels per year (i.e., channel load); and the marketing plans for individual services 2) Omnibus surveys – includes, e.g., Citizens First and Taking Care of Business studies, EKOS' Information Highway studies, Ipsos-Reid research, NFO Interactive surveys, TBS GOL Internet Research Panel |
|
Summary of results achieved in 2003 |
Mixed – Client take-up of on-line federal services is increasing. Visits to the Canada Site and two of the three gateways increased noticeably in 2003. Some well-established GOL services now have more than 50% of their clients completing transactions on-line. Overall, the current take-up for transaction services (24%) is comparable to self-reported use of on-line banking in Canada (21-32% according to EKOS (2003)), a reasonable benchmark of what to expect. However, when CCRA's "Individual Tax" is excluded, the average take-up rate drops considerably. Further, a majority of Canadians think that the federal government does not do a good job of marketing its on-line services, and are only moderately aware of what is available. Only a few GOL services have marketing plans for 2004 and/or take-up strategies. While data quality improved significantly in the past year, there are still gaps and inconsistencies in measurement methodologies that make it difficult to set realistic take-up targets. |
|
Raw data |
Trends in overall Internet use
– More than 51% had at least one person who regularly used the Internet from home, up slightly from almost 49% in 2001
– 21% do all of their banking through the Internet – The number of experienced "net bankers" is increasingly common; 15% of all Canadians, and 48% of those who bank on-line, have at least two years of experience
– In 2002, 87% of households whose head possessed a university degree had at least one regular Internet user, compared with 66% whose head possessed a high school/college education, and 30% whose head had less than a high school education – In 2002, 88% of households earning $70,000 or more had at least one regular Internet user, compared with 75% of those earning $40,000-69,999, 51% of those earning $23,001-39,999, and 33% of those earning less than $23,000
– The percentage of those 65 and older using the Internet also increased noticeably Awareness of federal service delivery on-line
– 15% of Internet users rate themselves as very aware, 67% as moderately aware, and 15% as not at all aware (three percent say they do not know)
– Only 17% of this group (four percent of all Canadians) identified the correct URL, Canada.gc.ca (or gc.ca)
– Note that self-rated awareness in the EKOS survey is lower than in the NFO Interactive survey (mentioned above); one reason for this may be that the difference in the respondents – EKOS surveyed all Canadians while NFO Interactive surveyed Internet users, who as a group are likely to have more knowledge of what is available on-line than Canadians in general; another reason may be the difference between the questions – EKOS focused on federal services, while NFO Interactive asked about government services in general
– Six of the 45 services (13%) that are complete have marketing plans for 2004 Use of gateways and portals
– A visit is defined as "the sequence of interactions an Internet user has on a Web site without 30 minutes of inactivity"
– E.g., page views for the Public Safety portal increased by 144% from June-September 2003 in comparison with the same period in 2002; and the number of page requests for the Persons With Disabilities portal in July 2003 were more than double the number of requests in January of the same year (that is, prior to the addition of portal links on the Canada Site)
Take-up of GOL services – channel volume
– 41% say their most recent contact was by phone, about the same as in 2002 (42%), 17% by mail, down from 21%, seven percent in-person, down slightly from nine percent
– 81% say that their most recent contact involved the phone, 24% in-person, 10% mail, nine percent fax, six percent a comment form on a Web site, and three percent a public kiosk
– 58% of businesses say they use the mail vs. 25% of individuals, 45% say they visit government Web sites vs. 30% of individuals, 35% say they use the fax vs. five percent of individuals, and 25% visit government offices vs. 48% of individuals
– Similarly, a recent international benchmarking study found that 51% of Canadians have used on-line federal services/Web sites in the past year; overall, Canada currently ranks #6 in the world in terms of use of on-line federal services – the global average is 30% (TNS, 2003)
– Information services are mostly responsible for this increase, with the volume of interactions growing by 88% (from 145 million interactions in 2002 to 280.6 in 2003)
– The volume of interactions through IVR decreased from 79.4 million in 2002 to 70.1 million interactions in 2003, while the other channels remained fairly stable Take-up of GOL services – where clients complete transactions
– 27% of government clients (either from federal departments or agencies, or from other jurisdictions) completed transactions on-line, up from 17%, 25% of individual Canadians, up from 22%, 19% of businesses, up from 16%, and two percent of international clients completed transactions on-line – One reason for the difference in take-up between Canadians and businesses is that large services for Canadians such as individual tax filing and job searches/applications have been available for several years, while comparable services for businesses such as tax filing have just been launched; however, departments and agencies expect businesses will significantly outpace Canadians in their use of on-line services over the next few years
– E.g., HRDC's "Employment Services", 97-100%; CCRA's "Commercial Customs", 97%; Jus.'s "Family Orders and Agreements Enforcement Assistance", 90%; PWGSC's "Linguistic and Multicultural Services", 90%; Public Service Commission's (PSC's) "Recruitment", 87%; Canadian Firearms Centre's (CFC's) "Canadian Firearms Program" (registration applications), 65%; DFAIT's "Trade Commissioner", 57%; and INAC's "Indian Registration/Certificate of Indian Status", 50%
– When on-line job services are removed as well (DND, HRDC, INAC, PSC), this percentage further decreases to seven percent in 2002 and 12% in 2003 A note on data quality
– When CCRA's "Individual Tax" is removed from this subset of mature services, the percentage of clients completing transactions on-line in 2003 decreases to 15%
|
|
Plans for improvement |
– The marketing strategy will focus on three key themes: increased awareness (e.g., broad advertising campaigns in all media to inform citizens and businesses that the federal government has on-line services for them), maximum impact (e.g., coordination of marketing plans, including identification of common themes, audiences and opportunities), and targeted promotion of services
|
|
|
Client Satisfaction |
|
Indicator(s) |
Increased satisfaction levels through high quality services that provide clients time, effort, and cost savings relative to other delivery channels Robust and accurate citizen/client feedback drives the improvement and evolution of services |
|
Measurement Level and Technique |
Measurement at the service and "whole of government" level primarily using citizen/client feedback |
|
Primary tool(s)/data source(s) |
1) Departmental reporting on GOL plans and progress – the reporting includes three types of data related to client satisfaction: data from client satisfaction surveys; self-assessments of client benefits; and information on service standards 2) Omnibus surveys – includes, e.g., Citizens First and Taking Care of Business studies, TBS GOL Internet Research Panel |
|
Summary of results achieved in 2003 |
Mixed – While overall client satisfaction is relatively high, there are neither enough surveys of client satisfaction by departments and agencies nor sufficient use of satisfaction results to communicate with clients and develop service improvement strategies. Canadians' overall satisfaction with federal services has increased from 60% in 1998 to 64% in 2003. Half of the GOL departments and agencies have recently done a client satisfaction survey for at least one of their services; the average score is four or "satisfied" (on a one to five scale ranging from "very dissatisfied" to "very satisfied"). A smaller number have done surveys specifically for the Internet delivery channel. Only a few GOL services have sought their clients' perceptions of expected client benefits such as easier access and better information. Only one-third of departments and agencies have published the results of their satisfaction surveys, and a similar number have published service standards (two-thirds report having established service standards). |
|
Raw data
|
– Canada Post, CCRA tax services, HC information services, and HRDC employment services all contributed to this improvement, as did Canadians' satisfaction with on-line service generally
– Satisfaction with on-line government services is 68%, whereas satisfaction with services available by phone is only 56% – More than 80% who have used the on-line version of a government service would do so again – Over 50% of Canadians who want to receive government services through a different delivery channel would like to switch to the Internet or e-mail; this compares with only 21% who would like to switch to in-person service, 19% to the phone, and seven percent to mail – Greater satisfaction with on-line service delivery is noteworthy, as 51% of Canadians (66% of Internet users) have visited a federal Web site at least once in the past year, up noticeably since 2001 (EKOS, 2003); more detailed results can be found in the take-up section of this report
– 82% of businesses that have used the telephone to receive a government service would do so again (their strongest allegiance), while 73% of individuals would use it again – In contrast, 78% of businesses that have used the on-line version of a government service would do so again, while 81% of individuals would use it again (their strongest allegiance) GOL services – satisfaction surveys
– 14 of these 45 services, covering 10 of 30 departments and agencies, report publishing survey results on their Web sites
– Examples of services that do use survey results in this way include: HC's "Health Promotion", IC's "Federal Economic Development Initiative in Northern Ontario (FedNor)", NRCan.'s "Geographical Applications", Stats. Can.'s "Collection", "Communications and Dissemination", and "Stakeholder Relationships"
– These results are consistent with findings from a recent omnibus survey, which reports that average satisfaction score is about four or "satisfied" for federal Web site usage (where one is "very dissatisfied" and five is "very satisfied") (GOL Internet Research Panel, 2004) – 13 of these services have compared their Internet channel with other channels; most (nine) indicate that satisfaction with on-line delivery is the same as with other channels GOL services – service standards
– 32 of these 56 services also report having done a client survey in the past two years – 26 services, covering nine of 30 departments and agencies, have published their standards GOL services – benefits
– 115 of 130 services (88%) are providing easier access – 62 services (47%) are providing time savings – 62 services (47%) are providing simplified interactions with the federal government – 60 services (46%) are providing better information leading to increased education, skills, and professional development opportunities, and/or better information leading to increased business opportunities – 41 services (31%) are bundling related components in ways that are not possible through traditional delivery channels (e.g., in-person, phone, mail) – 24 services (18%) are providing cost savings
– Actual feedback is positive, with elements such as Web site clarity and ease of navigation typically rated as "good" to "very good"
– Actual feedback is mixed; some clients are very satisfied with the information currently on-line; others think that it is incomplete or does not fully meet their needs |
|
Plans for improvement
|
|
|
|
Security |
|
Indicator(s) |
Use of the common infrastructure Adequate steps by departments and agencies to ensure that transactions are secure Citizens/clients perceive that on-line services are secure |
|
Measurement Level and Technique |
Measurement at the "whole of government" level using citizen/client feedback, and at the service level using self-assessments, client feedback, and tracking of how they engage the federal government |
|
Primary tool(s)/data source(s) |
1) Omnibus surveys – includes, e.g., EKOS' Information Highway studies 2) Secure Channel roadmap, outlining plans for the provision and use of Secure Channel services 3) Departmental reporting on GOL plans and progress – the reporting includes two types of data related to security: data on what type of IM/IT infrastructure is required by which date; and information on threat risk assessment, certification and accreditation, and business continuity planning 4) Discussions among departmental security coordinators and departments and agencies with a lead role in this area |
|
Summary of results achieved in 2003 |
Mostly positive – Canadians have more confidence in governments than either the banks or the private sector in terms of offering safe on-line services. However, they have significantly more comfort in applying on-line for a program or service than for making an e-payment. The current take-up of on-line federal services also provides evidence of Canadians' increasing confidence – the proportion of clients who completed transactions on-line increased from 21% in 2002 to 24% in 2003. In general, departments and agencies are meeting the security expectations of their clients. A reliable "whole of government" incident handling process is in place to respond to external threats. The Secure Channel now offers authentication and e-payment services, both of which Canadians and businesses are using in increasing numbers. Secure Channel is developing and implementing services for which there is strong departmental demand, enabling the federal government to conduct secure on-line transactions with clients. Departments and agencies are at various stages in implementing the Government Security Policy, which requires threat and risk assessments and business continuity plans for federal services. |
|
Raw data
|
Perceptions of on-line service delivery
– 62% agree (20% disagree) that governments would not offer the choice of doing things as filing taxes through the Internet unless it was safe to do so, up from 59% in 2002 – In contrast, 59% agree (24% disagree) that banks would not offer the choice of banking on-line unless it was safe to do so, and 38% agree (39% disagree) that companies would not offer the choice of doing things such as buying products through the Internet unless it was safe to do so
– Internet users have a higher level of comfort than Canadians in general – 60% of Internet users have a moderate (32%) to high (28%) level of comfort with providing their credit card number on-line in order to make a payment to the federal government Security coordination
– Secure Channel Information Protection Centre (IPC) – PWGSC IPC – PSEP (formerly OCIPEP) Coordination Centre – Cyber Incident Coordination System triage unit – Alerts and advisories from PSEP and other government and non-government organisations
The common, secure infrastructure
– SCNet, which is advanced IP/VPNe network inter-connecting 130 departments and agencies, certified to "Protected A" and scaleable to "Protected B" (see the end of this section for definitions) – Security and authentication services, including ePass, which control access to on-line applications, protect privacy, and secure IT infrastructure from hackers – Payment services
Government Security Policy
– There is some evidence suggesting that there is a lack of understanding of what is expected and, thus, that an awareness program and/or direct support may be required; training is either being set up, or is available for federal employees – Note that the Secure Channel has completed its TRA, BCP, and C&A
|
|
Plans for improvement
|
– Enhancing the Secure Channel IPC – Building interoperability and coordination capabilities among the Secure Channel IPC, the PWGSC IPC, and PSEP (formerly OCIPEP) – Developing common analysis services and solutions with an infrastructure overlay that will assist departments and agencies in establishing or enhancing their own detect and respond capabilities – Developing an intrusion detection standard – if passed, Bill C14 will give departments and agencies the legal framework for intrusion detection, acknowledging the responsibility of the federal government both to securely manage their IM/IT assets and to respect individual privacy rights (note that while some guidance is currently available, the standard will not be finalised until Bill C14 is passed)
– Detection, Analysis, and Response Infrastructure Model, led by PSEP, which will provide a comprehensive, secure infrastructure to support the detection, analysis, and response to cyber incidents within the federal government – Threat and Vulnerability Analysis System, led by CSE, which will support the identification of and response to current and emerging safety and security cyber threats, and improve the monitoring and mitigation of risks to federal services – Cyber Incident Coordination System, led by PSEP, which supports the coordination of incident management across the federal government
|
Terminology
"Protected A" – if its compromise could reasonably be expected to cause a low level of injury to private or non-national interests, for example, disclosure of an exact salary figure
"Protected B" – if its compromise could reasonably be expected to cause a medium level of injury to private or non-national interests; such information concerns an individual or an organisation, and is considered to be particularly sensitive
Security risk management
1. Threat and Risk Assessment (TRA), including a Statement of Sensitivity (SOS) identifying and categorising assets in terms of their confidentiality, integrity, availability, and value based on the Operational Standard for the Identification and Categorisation of Assets
2. Business Continuity Plan (BCP), including a Business Impact Assessment (BIA) documenting the relative priority/criticality of services and systems, and the maximum/minimum allowable down-time
3. Certification and Accreditation (C&A) of systems and services, confirming that the security requirements have been fulfilled, and signifying that management has authorised the system or service for operation
4. Vulnerability Assessments, which identify inherent vulnerabilities and recommend remedial action
5. Active Defence, one based on the Protect-Detect-Respond-Recover cycle
|
|
Privacy |
|
Indicator(s) |
Adequate steps to protect personal information/individual privacy on-line Citizens/clients perceive that on-line services offer good privacy protection |
|
Measurement Level and Technique |
Measurement at the "whole of government" level using citizen/client feedback, at the departmental level using self-assessments, and at the service level using self- and third-party assessments |
|
Specific tool(s)/data source(s) |
1) Omnibus surveys – includes, e.g., EKOS' Information Highway studies, Ipsos-Reid research 2) Privacy Impact Assessments (PIAs) – federal institutions conduct PIAs for new or redesigned programs/services that raise privacy issues; they provide the results of these assessments to the Privacy Commissioner before implementation, and publish summaries of the results on their Web sites 3) Departmental reporting on implementation of the Common Look and Feel Standards for the Internet (CLF) – federal institutions self-assessed their compliance with the seven CLF components (33 standards in total) using either all or a representative sample of their Web sites; the score for each component is an average of what all departments and agencies subject to CLF have reported; the focus, here, is reporting on consistent placement and use of important notices (especially privacy notices) (CLF component descriptions, related standards, and a self-assessment guide are available at: http://www.tbs-sct.gc.ca/clf-nsi/index-eng.asp) |
|
Summary of results achieved in 2003 |
Very positive – A high percentage of Canadians think that the federal government has the appropriate safeguards in place to protect personal information submitted on-line; they have as much confidence in the federal government as the banks, and more confidence in the federal government than the private sector to protect this information. While many Canadians still have concerns about on-line transactions that require the exchange of personal information, the current take-up of federal services (from 21% in 2002 to 24% in 2003) provides evidence that these concerns are being addressed. Departments and agencies are ensuring that protection of privacy is a key consideration from the initial design of on-line services and onwards; to accomplish this, they are conducting an increasing number of privacy impact assessments. While few Canadians understand the overall privacy stance of the federal government, including the extent to which personal information is being shared among departments and agencies, a majority believe that the privacy law is being followed. |
|
Raw data
|
General perceptions
– In contrast, 48% do not mind banks, and 46% do not mind companies (down slightly from 50% in 2001), using personal information as long as they know about it and can stop it
– 60% are comfortable with some sharing of personal information among jurisdictions Perceptions of on-line service delivery
– About the same percentage of Internet users share this concern
– The information submitted includes: name (75%), e-mail address (74%), address (57%), home phone number (48%), credit card number (37%), income data (20%), and social insurance number (16%) Perceptions of the federal government
– 74% are somewhat (59%) to extremely (15%) confident in the federal government's ability to fully protect personal information submitted on-line – In contrast, 73% are somewhat (60%) to extremely (13%) confident in the banks' ability, and 62% are somewhat (52%) to extremely (10%) confident in credit card companies' ability, to fully protect personal information submitted on-line Privacy Impact Assessment (PIA) process
– In his 2002-03 Annual Report to Parliament, the interim Privacy Commissioner noted that while the first PIAs submitted did not contain complete information, their quality is improving as departments and agencies gain more experience in using them
Common Look and Feel Standards for the Internet (CLF) – Important Notices
|
|
Plans for improvement |
|
|
|
Efficiency |
|
Indicator(s) |
Return on investment (ROI), cost avoidance, and operational efficiencies from on-line service delivery |
|
Additional explanation of what is being measured |
The data below are only indicative of cost savings, cost avoidance, and increased productivity to date. A full evaluation of mature GOL services would be necessary in order to provide conclusive results. Many experts believe that it will take time to realise cost savings, as on-line service is a new delivery channel, and as departments and agencies have not closed existing delivery channels. It should be noted as well that the initial GOL objective was not to achieve a specific cost savings amount, but to improve the service quality and improve client satisfaction by making federal services available through the Internet. |
|
Measurement Level and Technique |
Measurement at the service and "whole of government" level using self-assessments |
|
Primary tool(s)/data source(s) |
1) Departmental reporting on GOL plans and progress – the reporting includes two types of data related to efficiency: productivity, cost savings, and cost avoidance; and impacts of on-line service delivery on client requests, physical points of presence (in-person, call, and mail centres), and human resources |
|
Summary of results achieved in 2003 |
Mixed – Almost half of GOL services are realising some combination of cost savings and cost avoidance; many are reinvesting savings to improve service delivery. As well, about two-thirds of these services are realising productivity benefits, allowing employees to focus on more complex and higher-value activities in response to client needs and requests. However, there is little evidence that GOL departments and agencies have set aggressive channel migration strategies (and take-up targets), where appropriate, in order to maximise savings from lower cost on-line transactions. Savings are mostly through less use of paper and postage resources, and typically are not significant in terms of amount. Further, there is a lack of detailed costing, departmental benefits, and channel impact data necessary for the effective management of service delivery across all channels. |
|
Raw data
|
– 78% of transaction services report these benefits, as compared with 62% of information services – 92% of mature transaction services ("levels 7-9" using the model discussed in the critical mass of services section) report these benefits
– For further information, see the critical mass of services and take-up sections of this report Cost savings and avoidance
– 28 of these 59 services report cost savings, 12 cost avoidance, and 19 both savings and avoidance – 16 services report a net dividend – Many services report that they either reinvest savings in order to improve service, or use them to offset higher technical and administrative costs
– Based on filings, however, there is no correlation between realising cost savings (gross or net) and maturity of on-line transactional capability
– Correspondingly, only six of the 47 services reporting cost savings (13%) also report that the number of in-person, mail, and/or call centres decreased in 2003 Productivity
– About two-thirds of these services point to greater client self-service as the reason for increased productivity; about one-third specify that client self-service is allowing employees to devote their time to other, higher-value activities – 81% of the services reporting that client enquiries are becoming more complex also report increased productivity benefits
– 20 of these 27 services (74%) are transactional Channel management
– In addition, 10 services (nine departments and agencies) report that they will decrease the number of in-person centres by 2005 in response to decreasing demand for over-the-counter service
– All services reporting that the number of call (and in-person) centres will increase by 2005 also report that client enquiries will be more complex
– Some transaction services are predicting that the volume of interactions through the Internet will increase more than what calculated trends based on data from previous years would suggest; correspondingly, they are overestimating future decreases in in-person (and telephone agent) volume – As well, some information services are predicting that the volume of interactions through the Internet will not increase as much as what calculated trends based on data from previous years would suggest – These data may suggest that transaction services need to implement more aggressive channel migration strategies to encourage clients to use self-serve channels, and that information services may need to ensure that they can handle the increasing volume on the Internet channel (note however, that these findings may in part be the result of the smaller sample size used for the analysis – only services providing predictions for 2005 were used for the comparison) – More detailed results can be found in the take-up section of this report |
|
Plans for improvement |
|
Innovation |
|
|
Indicator(s) |
Use and demonstration of innovative Internet applications – establishment of Canada as a leader in the knowledge-based economy and society |
|
Additional explanation of what is being measured |
The focus of this dimension of the GOL measurement regime is on information and communication technology (ICT) innovations and their economic impacts – on award-winning on-line services, "readiness" for e-commerce, and e-procurement/e-sales activities. At the same time, it is necessary to recognise that the GOL initiative has only an indirect impact on e-commerce and economic growth. These results are also influenced by global factors such as the political climate, and by country-specific factors such as employment levels, real disposable income, and educational attainment. (Note: broader innovation results can be found in Canada's Performance, report of the President of the Treasury Board to Parliament: /report/govrev/03/cp-rc1_e.asp) |
|
Measurement Level and Technique |
Measurement primarily at "whole of government" level using third-party assessments |
|
Primary tool(s)/data source(s) |
1) World Economic Forum (WEF) Networked Readiness Index – benchmarks the degree to which countries are capable of using technology to promote economic growth 2) Economist Intelligence Unit (EIU) e-Readiness Index – benchmarks the degree to which countries are prepared to take advantage of, and can maximise, Internet-based commercial opportunities 3) Stats. Can.'s Survey of Electronic Commerce and Technology – collects data on business/government Internet use and sales (with or without on-line payment) 4) GTEC awards (http://www.gtecweek.com/) 5) Omnibus surveys – includes, e.g., EKOS' Information Highway studies |
|
Summary of results achieved in 2003 |
Mixed – Only a minority of Canadians think that governments (federal, provincial, territorial) are developing and implementing innovative ICT. However, about one-third of GOL departments and agencies win awards for their use of technology in service delivery each year. Internationally, Canada ranks in the top 10 in surveys benchmarking countries' readiness to take advantage of Internet-based commercial opportunities. There is also initial evidence that Canada is beginning to realise this potential for economic growth through the use of ICT. |
|
Raw data
|
– Stats. Can. defines innovation as "a new or significantly improved product offered to clients, or a new or significantly improved process" General perceptions
– In contrast, 52% think that banks are on the leading edge in implementing new technology 2003 GTEC medallists (federal)
– Within this group, Canada ranks #9
– Canada ranks #4 for the ICT environment component of this index – it is particularly strong in areas such as availability of scientists and engineers, overall infrastructure quality, and number of secure Internet servers – Canada ranks #5 for the ICT readiness component of this index – it is particularly strong in areas such as availability of mobile Internet access, availability of broadband access, firm-level innovation, and business Intranet sophistication
– Canada ranks #10 for the ICT use component of this index – it is particularly strong in areas such as use of on-line payment systems, use of the Internet for coordination with customers and suppliers, presence of wireless business applications
– According to EIU analysis, "there appears to be a high degree of uniformity among top-rated countries: only 0.47 points (out of a possible 10) separates first place from tenth" – Canada ranks #1 on two dimensions of this index, business environment (e.g., strength of the economy, regulatory environment, competition policy, quality of infrastructure) and supporting e-services (e.g., availability of e-business consulting and technical support services, industry-wide standards for platforms and programming languages) – Canada is weakest on the connectivity dimension of this index (e.g., mobile-phone penetration, level of competition in the telecom industry) Impacts of ICT diffusion on e-commerce
– The number of experienced "net bankers" is increasingly common; 15% of all Canadians, and 48% of those who bank on-line, have at least two years of experience – "Net bankers" are doing more than half of all their banking on-line
– E-commerce in 2002 accounted for only 0.6% of total private sector operating revenue – Large firms are responsible for most e-commerce activity – E-commerce remains volatile; in 2002, seven firms stopped selling through the Internet for every 10 that started
– The proportion of businesses buying goods and services through the Internet increased from about 22% in 2001 to about 32% in 2002 – The proportion of government organisations buying goods and services through the Internet increased from about 55% in 2001 to about 65% in 2002
|
Appendix #1: Expected Outcomes
The current expected outcomes incorporate, and are broader in scope than, the original set articulated in 2000.
For convenience, the quality of the search capability on federal Web sites (first identified in 2000) is being tracked through client surveys; and access to integrated services as well as greater personalisation through integrated Web portals will be measured in greater depth and consistency through the accountability framework that these portals are developing. Note that individual examples of integrated services –like the interactive tool on the Doing Business with Canada portal for information about doing business with Canada, the diagnostic on the Exporting/Importing portal testing a company's export readiness, the interactive tool on the Financial Benefits portal identifying the benefits for which Canadians might be eligible, and the diagnostic on the Seniors portal providing personalised information for Canadians experiencing either the death of a parent or a life care transition – as well as greater personalisation – such as the password-protected customised page of links to federal information and services as well as the personalised e-mail service available through the Canada Site – are also tracked and reported through the annual report on the GOL initiative that is tabled in Parliament.
For credibility, the quality of information principles (first identified in 2000) are being measured through a self-assessment tool, the IM Capacity Check. For critical mass of services, the ability to communicate electronically with clients and the level of functionality of on-line services are being measured through the service maturity model; the benchmark for electronic communication is "level 5" and beyond on the service maturity model, and for apply, file, enquire, and payments functionality it is "levels 6-9" of the model specifically for transaction services. For service transformation, partnerships with other jurisdictions are being tracked through departmental reporting (at the service and funded project level) on GOL progress. For security, progress towards a secure on-line transactional capability is captured through the broader Secure Channel roadmap, which also identifies some of the leading-edge technologies being developed in support of integration.
|
Current expected outcomes |
Original expected outcomes (articulated in 2000) |
|
Convenience
|
|
|
Accessibility
|
|
|
Credibility
|
|
|
Critical mass of services
|
|
|
Service transformation
|
|
|
Take-up
|
|
|
Client satisfaction
|
|
|
Security
|
|
|
Privacy
|
|
|
Efficiency
|
|
|
Innovation
|
|
Appendix #2: Service Maturity Model
Maturity model – information services (developed by TBS)
|
Publish basic information (push) |
Note: "Basic documents" can include policies, relevant regulations and/or laws. They can also include reports, communications pamphlets, FAQs, etc., depending on the service in question. The key point to note is that these are documents that someone familiar with the service in question can easily and quickly identify, and that clients can easily obtain through off-line sources as well (e.g., offices, reference sections in libraries).
Note: "Complete", here, is understood only from the perspective of the service provider, not the client; and it focuses on documents that clients can easily obtain through off-line sources, not on the vast information holdings of services that could potentially be made available through new technologies. |
| Customise information holdings (push/pull) |
Note: There is a clear distinction between the third and fourth stages of progress. In this fourth stage, there is a strong client focus, which guides decisions in making information available on-line. Services have also made clear progress in going beyond providing the "basic documents" of stage three. Depending on their clients, and on the type of information that they provide, services can adopt different strategies as they provide more information on-line. They can treat information as a free public resource, and thus make it as comprehensive as possible; or they can see themselves as an authoritative source of information, and thus provide it more selectively. Note, however, that the "deepening of holdings" is an open-ended process; once begun, there is no end-point in putting information on-line. |
Note: "Segmentation" is more than just identifying clients using broad categories like individuals and businesses.
Note: The above list of interactive tools is not intended as a checklist – it is neither exhaustive, nor may every example be appropriate for every service. The key criterion in determining whether services have "a full range of interactive tools" is what clients can reasonably expect to do on-line, given both their information needs and current technological possibilities. |
|
Provide client-defined access to information (pull) |
|
Maturity model – transaction services (multiple versions in use in different jurisdictions and organisations, e.g. Accenture)
|
Publish (passive/passive) |
|
|
Interact (active/passive) |
|
|
Transact (active/active) |
|
Appendix #3: Transformation Model
(Developed based on service vision work for Canadians, businesses, and international clients)
|
1. Grouping and rationalising of related information/transactions to facilitate access |
|
Components of this stage include:
|
|
2. Development of interoperability across organisations and/or channels to personalise service |
|
Components of this stage include:
|
|
3. Integration in order to offer bundled solutions |
|
Components of this stage include:
|
|
4. Provision of a complete and "seamless" service experience (differentiated offerings/common enablers) |
|
Components of this stage include:
|