Treasury Board of Canada Secretariat
Symbol of the Government of Canada

ARCHIVED - Systems Under Development (Audit Guide) - March 1, 1991


Warning This page has been archived.

Archived Content

Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.


Appendix D: Audit Program for the General Design Stage

Stage: 3. General Design

Objective: 3.A To ensure that the general design of the system expands on the findings of the feasibility study, produces a functional description of manual and EDP processes, and devises an overall system design that can be used to obtain a commitment for further development.

Criterion: 3.A.1 System specifications are addressed in a System Specifications Report or similar document.

Audit Step: 3.A.1.1 Has a Systems Specifications document been prepared and released?

Y N N/A Comments XREF
         

Audit Step: 3.A.1.2 Verify that it contains at least the following:

  • system objectives and scope
  • general system concept and design considerations
  • chart showing function structure in terms of processes
  • logical data flow diagram showing flow among processes and data stores at the data element level
  • process descriptions, including complete and detailed definitions of processes for all business cases involved. Descriptions will include algorithms and validity checks
  • system interfaces: definitions at the data element level
  • system inputs and outputs: definitions at the data element level with the medium to be used for input and output specified
  • data stores: definitions of logical data stores at the data element level
  • volumes, timings, highs and lows, and quality specified for inputs, outputs, and data stores
  • service levels: Complete description of performance requirements. This will be used in later stages to confirm the technical feasibility and resources requirement of the system
  • audit, control, and security requirements - implementation requirements, including conversion
  • (see Gane and Sarson reference, Item 22 in Appendix I, for a description of some of the terms.)
Y N N/A Comments XREF
         
Criterion: 3.A.2 The accuracy and completeness of system specifications has been acknowledged by the appropriate level of user and by Data Processing management.

Audit Step: 3.A.2.1 Has the System Specifications document been reviewed by the Steering Committee/Sign Off Authorities? Have they signified acceptance of the need to continue the project? Note any conditional acceptance for follow-up in later stages.

Y N N/A Comments XREF
         
Criterion: 3.A.3 The data dictionary/directory has been updated to reflect the contents of the System Specifications document.

Audit Step: 3.A.3.1 Has the data dictionary/directory been updated to contain the system specifications?

Y N N/A Comments XREF
         
Criterion: 3.A.4 All required skills are still available to the project.

Audit Step: 3.A.4.1 Do the skills of the staff being employed on the project (as Team Members or Steering Committee/Sign Off Authority members) continue to meet the requirements specified in the Personnel Skills Summary?

Y N N/A Comments XREF
         
Criterion: 3.A.5 Dates for committee meetings and the items to be discussed at each meeting continue to be addressed in a Steering Committee Meeting Schedule or similar document.

Audit Step: 3.A.5.1 Has a Steering Committee Meeting Schedule document been prepared and released to all interested parties, including EDP and user management?

Y N N/A Comments XREF
         

Audit Step: 3.A.5.2 Attend or review the minutes of the committee meetings and note the following:

  • representatives from EDP and user management were represented at each meeting, and
  • meetings were held regularly.
Y N N/A Comments XREF
         
Criterion: 3.A.6 The status of the project compared to the budget and schedule contained in the Feasibility Stage Status document has been addressed in a General Design Stage Project Status Report or similar document.

Audit Step: 3.A.6.1 Has a General Design Stage Status document been prepared and released?

Y N N/A Comments XREF
         

Audit Step: 3.A.6.2 Verify that it contains at least the following:

  • actual resources used to date, compared to planned, with reasons for variance
  • actual milestones achieved to date, compared to planned, with reasons for variance
  • detailed plan for the Detailed Design Stage, including the following activities:
    • updating the data dictionary/directory
    • carrying out the final design of all inputs and outputs
    • developing a detailed implementation plan - verifying that security concerns have been met
    • developing a detailed testing plan
    • estimating performance and resource requirements
    • updating project plans and budgets
    • updating the cost/benefit analysis
    • obtaining management approval
  • the preliminary plan for the Implementation Stage, includes the following:
    • identification of manual procedures to be developed
    • manuals that will be affected
    • facilities needs
    • communications needs
    • training
  • an updated budget and reasons for any changes
  • an updated schedule and reasons for any changes
  • an updated cost/benefit analysis
  • a recommendation to continue or discontinue the project
Y N N/A Comments XREF
         

Audit Step: 3.A.6.3 Verify actual resources used in source documents.

Y N N/A Comments XREF
         

Audit Step: 3.A.6.4 Are the updated budget and schedule in keeping with the updated cost/benefit analysis?

Y N N/A Comments XREF
         

Audit Step: 3.A.6.5 Verify the updated cost/benefit analysis against the cost/benefit analysis from the previous stage and from source documents.

Y N N/A Comments XREF
         

Audit Step: 3.A.6.6 Determine that the updated cost/benefit analysis has taken into consideration the human resource impact requirements.

Y N N/A Comments XREF
         
Criterion: 3.A.7 The accuracy and completeness of the General Design Stage Status document and agreement with it has been acknowledged by the appropriate level of user and by Data Processing management.

Audit Step: 3.A.7.1 Has the General Design Stage Status document been reviewed by the Steering Committee/Sign Off Authorities and have they signified acceptance of it?

Y N N/A Comments XREF
         
Criterion: 3.A.8 A human resources impact analysis is planned.

Audit Step: 3.A.8.1 Does the detailed plan for the Detailed Design Stage take into consideration a human resources impact analysis? Does the planned analysis cover all personnel to be affected? i.e. those to be trained for new system and those to be re-deployed.

Y N N/A Comments XREF
         

Audit Step: 3.A.8.2 Does the detailed plan for the Detailed Design Stage take into consideration the marketing of the new system? i.e. communicating to all those affected, the impact of the system on the department and themselves.

Y N N/A Comments XREF
         

Objective: 3.B To establish that data processed and stored by the system will be complete, accurate, and authorized.

Criterion: 3.B.1 Processing control techniques have been outlined in a System Processing Controls Specifications or similar document.

Audit Step: 3.B.1.1 Has a System Processing Controls Specifications or similar document been prepared and released?

Y N N/A Comments XREF
         

Audit Step: 3.B.1.2 Verify that it addresses at least the following (see Appendix for further references to data controls):

  1. Completeness
    1. There should be some method of ensuring that all data are initially recorded and identified.
    2. Control should be established close to the source of the transaction.
    3. Output should be reconciled to input.
    4. There should be some method of ensuring that corrections for all identified errors are re-entered into the system.
    5. The timing of input submissions and output distribution should be properly coordinated with processing.
    6. Procedures are needed to ensure that output reports are independently reviewed for completeness and form.
  2. Accuracy
    1. Procedures should exist to prevent errors in the preparation of input or source data, and to detect and correct any significant errors that do occur.
    2. Procedures should exist to prevent errors arising when data are converted to machine processable form, and to detect and correct any significant errors that do occur.
    3. There should be procedures to ensure that data are transmitted accurately to the computer centre.
    4. Procedures should ensure that only valid files are used.
    5. Controls must ensure that the accuracy of data is maintained during processing.
    6. Procedures should ensure that program computations are performed correctly.
    7. There should be a system of control over the physical operations of the computer system.
    8. Procedures should exist to ensure that all significant errors that have been identified at various stages in the system have been corrected, re-entered and properly reflected in the output.
    9. Procedures are needed to ensure that all required output reports are delivered to the proper user departments.
  3. Authorization
    1. To ensure that only authorized data is processed.
    2. Security, privacy, and accessibility level classifications (see 2.B.2.1) for data related to the system should be determined and appropriate measures devised to ensure proper storage, transmittal, access, privacy and destruction.
    3. There should be some method of identifying and locating the component file records and input/output documents involved in the processing of a given transaction or in the accumulation of a given total.
  4. Backup/Recovery
    1. Procedures for system backup/recovery should be documented and related training plans prepared.
    2. Procedures for data preparation, transcription, data control, and output distribution should be documented and related training plans prepared.
  5. Audit Trail
    1. There should be some way to identify and locate the component file records and input/ouput documents involved in the processing of a given transaction or in the accumulation of a given total.

Note: Different control concepts apply to different types of systems (e.g. batch versus on-line). See the Bibliography in Appendix I for books on controls for various types of system.

Y N N/A Comments XREF
         
Criterion: 3.B.2 The accuracy and completeness of the processing control technique specifications has been acknowledged by the appropriate level of user and Data Processing management.

Audit Step: 3.B.2.1 Has the Processing Control Specifications document been reviewed by the Steering Committee/Sign Off Authorities? Have they signified acceptance of it? Note any conditional acceptance for follow-up in later stages.

Y N N/A Comments XREF
         

Objective: 3.C To ensure that the system will operate efficiently and effectively.

Criterion: 3.C.1 System management control techniques are outlined in a System Management Controls Specifications Report or similar document.

Audit Step: 3.C.1.1 Has a System Management Controls Specifications Report or similar document been prepared and released?

Y N N/A Comments XREF
         

Audit Step: 3.C.1.2 Verify that it addresses at least the following:

  1. Efficiency
    1. There should be a standard or set of standards to determine system efficiency.
    2. There should be a mechanism to compare performance with standards and to report variances.
    3. There should be procedures for managers to follow up on variances from standards and for recording action taken.
  2. Effectiveness
    1. Effectiveness standards for the system's objectives should be established.
    2. There should be a mechanism to determine and report situations where systems are no longer able to meet their original objectives.
  3. Economy
    1. Management should have formal procedures to review projects and their resulting applications system regularly for economy.
Y N N/A Comments XREF
         
Criterion: 3.C.2 The accuracy and completeness of the system management control technique specifications have been acknowledged by the appropriate level of user and Data Processing management.

Audit Step: 3.C.2.1 Has the System Management Controls Specification document been reviewed by the Steering Committee/Sign Off Authorities? Have they signified acceptance of it? Note any conditional acceptance for follow-up in future stages.

Y N N/A Comments XREF