Common menu bar links
Breadcrumb Trail
ARCHIVED - Systems Under Development (Audit Guide) - March 1, 1991
This page has been archived.
Archived Content
Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.
Introduction
Background
In late 1983, owing to the importance of the early output of the Task Force on Informatics, the Office of the Comptroller General decided to suspend publication of the preliminary version of this "Guide to the Audit of Systems Development Performance". The Task Force, established by Treasury Board on July 7, 1983, was expected to require 12 to 18 months to complete their job. During that period, however, a Policy Interpretation Notice (PIN) 1984-03 ("Pre-implementation Audit") was issued. That PIN defines the purpose and scope of this Guide.
The Task Force issued their report in 1985 and on July 22, 1986, an Information Management Policy Overview draft was issued by the Administrative Policy Branch of the Treasury Board Secretariat, partly in response to that Report. All these documents, while portraying the tremendous technological changes in the field of systems development, also underline the importance of the PIN's instructions about auditing systems development. The PIN states:
"Pre-implementation audits should be undertaken for all major systems under development in departments and agencies; they should be reflected in the departmental/agency internal audit policies and plans; and the potential loss of auditors' objectivity can be minimized through appropriate terms of reference and a suitable assignment strategy".
It is in the belief that management control over systems under development through the audit process is important that this Exposure Draft is offered.
Purpose of Guide
This Guide is written for the senior internal auditor conducting a Systems Under Development (SUD) Audit. A SUD Audit is defined as:
"A review and evaluation, at various stages in the systems development life cycle, of a selected system or large scale enhancement to an existing application. The audit includes a review of compliance with specified aspects of a department's systems development process and a review of the controls being built into the system to ensure completeness, accuracy, security, proper authorization and auditability of the data being processed."
The auditor should know that Information Technology audits, in addition to reviewing systems under development, can also evaluate the computer centre, post-implementation, on-going system, data dictionary, end user computing, data security, data management, Information Technology procurement, Information Technology management and any other type of audit project that may have an impact on issues that fall within a SUD audit. The preceeding are not objectives, they are areas for study. An audit of systems under development will examine the following (see PIN 1984-03):
- The project management and systems development process.
- Products reflect the control framework being designed in conjunction with (surrounding), or as an integral part of, the system under design.
Organization
In Chapter 1, we discuss the environment surrounding systems development in today's Public Service.
Chapter 2 provides a description and model of the systems development life cycle and the roles and responsibilities of the main players.
Chapter 3 set out objectives and criteria for conducting a SUD audit with reference to control, economy, efficiency, and operational effectiveness in each stage of the process. The chapter deals first with the five major activities of audit and how those activities relate to each of the seven system development life cycle stages. Each of the subsequent sections of chapter three then deals with project, data integrity, and systems management control objectives at each stage of the life cycle.
Appendix A contains a grid of suggested interviewees for each Detailed Criteria. Appendices B through H contain detailed criteria for each Systems Development Life Cycle Stage from Initiation through to Post-Implementation.
Finally, there is a bibliography in Appendix I, and a TB Policies and Standards listing in Appendix J.
The Value of the Systems Development Process
As stated in Policy Interpretation Notice 1984-3 on Pre-Implementation Audit:
"systems development projects are notorious for cost/time overruns; implemented systems are equally notorious for not meeting all user requirements; systems, particularly EDP systems, often have under-designed control frameworks; and recent cost-reduction programs ... have focused increased attention on improving the productivity/efficiency of all processes. This puts the spotlight particularly on the systems development process because of the costly down-stream effects of inadequate design and implementation."
When the investment in systems development and the dependence of departments on systems to manage and deliver their programs are both considered, the advantage of an early warning to management of any inadequacies in the systems development is clear. To this end, the existence of a formal departmental systems development life cycle provides essential standards for establishing management control over specific projects or major enhancements.
Reporting of the Audit of Systems Development
A SUD audit must take place as the system is being developed, not after the system has been implemented. In addition, the sooner the project developer is aware of audit findings, the easier it is for remedial action to take place. It is also axiomatic that solutions, to design or project management weaknesses, are more efficiently implemented the earlier, in the development process, that audit is involved.
In view of this, "Special Reporting Considerations" presents some detail methodology early in Chapter 3 (see figure 1).
Special Considerations
Auditing systems under development is different, from On-going and Post-implementation system's audit, in that one may "revisit" the same system's development up to seven times. Thus, much of the audit work accomplished in early stages of the development process becomes ground work for auditing in the later stages of development. Chapter 3 is written with this aspect in mind.
Figure 1: The Cost of Change
The importance of an audit concern for project activity to properly communicate human resource impact, and ensure that there are plans to cope with that impact, is covered in more detail at the start of Chapter Three and with control objectives in the Project Control (A) stream.
The auditor must also verify, early in the development process, that the project reflects departmental strategic planning and is directly related to senior management objectives. Project Control (A) Objectives (Chapter 3) are provided to deal with these points.
The auditor should consider the recommendation of the Verification and Validation contracting technique, based on a risk assessment, if the project is not using that technique. More detail is included early in Chapter 3.
The auditor's early involvement in the strengthening of controls may raise
questions about his or her objectivity in auditing the on-going system at some
much later date. This issue is discussed in more detail later in the report, but
it can be said here that the assignment of different auditors in the on-going
systems audit should adequately address this issue (see reference to PIN
1984-03, Page 1).