Program Evaluation - ANSI

To help employers improve their safety training programs, the American National Standards Institute (ANSI) issued ASSE/ANSI 490.1-2016, Criteria for Accepted Practices in Safety, Health, and Environmental Training. This module covers some of the important concepts in the standard for training evaluation.

Once a safety training program is established, it should be evaluated initially to verify that it is being implemented as intended. After that, employers should periodically, and at least annually, step back and assess what is working and what is not, and whether the program is on track to achieve its goals.

Whenever training program evaluations identify opportunities to improve the program, employers, managers, and supervisors, in coordination with employees, should make adjustments and monitor how well the program performs as a result.

Sharing the results of monitoring and evaluation within the workplace, and celebrating successes, will help drive further improvement.

As with other programs, safety training program evaluation and improvement includes:

• Establishing, reporting, and tracking goals and targets that indicate whether the program is making progress.
• Evaluating the program initially and periodically thereafter to identify shortcomings and opportunities for improvement.
• Providing ways for workers to participate in program evaluation and improvement.

Evaluating Program Effectiveness

To make sure that the training program is accomplishing its goals, an evaluation of the training can be valuable.

• Training should have, as one of its critical components, a method for measuring the effectiveness of the training.
• When course objectives and content are developed for a training program, a plan for evaluating the training session(s) should be designed and integrated into the program’s other elements.
• An evaluation will help employers or supervisors determine the amount of learning achieved and whether a worker’s performance has improved on the job.

Below are some methods commonly used to evaluate the effectiveness of training.

1. Employee opinion: Questionnaires or informal discussions with employees can help employers determine the relevance and appropriateness of the training program.
2. Supervisor observations: Supervisors are in good positions to observe an employee's performance both before and after the training and note improvements or changes.
3. Workplace improvements: The ultimate success of a training program may be best evaluated by looking at changes throughout the workplace that result in reduced injury or accident rates.

However you conduct the training evaluation, it can give employers the information necessary to decide whether employees achieved the desired results, and whether the training session should be offered again at some future date.

Kirkpatrick/Phillip's 5-Level Evaluation Model.

The Five-Level Evaluation Model

To make sure the training program is conforming to ANSI/ASSE Z490.1, we recommend a five-Level approach developed by Dr. Donald Kirkpatrick and modified by Dr. Jack Phillips. Back in 1954, Donald Kirkpatrick developed a four-level evaluation model and later, in 1997, Dr. Phillips added a fifth-level that addresses the impact of training on profitability.

Phillips states that after determining a learning program's business impact at Kirkpatrick's Level 4, you can translate that impact into monetary terms and compare it to the total cost of the program to calculate ROI. Today, the Kirkpatrick/Phillips model is the most widely used training evaluation approach in the world, and is expressed as five levels of outcomes as follows:

Kirkpatrick - Levels 1-4

1. Reaction - did the learner like the safety training?
2. Learning - did the training increase the learner's safety knowledge, skills, and abilities (KSAs)?
3. Application - did the learner successfully apply the KSAs to the job?
4. Results - did the safety training have a positive impact on business key performance indicators (KPIs)?

Phillips - Level 5

5. Return - did the company show a positive return on the training investment (ROI) expressed as a cost/benefit ratio?

The first three levels of training evaluation focus on the impact of training on the learner. The four and fifth levels evaluate the impact safety training has on the company. We'll discuss each of the five levels in more detail in Module 7.

ANSI Z490.1 Guidelines for Evaluating Training Programs, Process, and Results

ANSI Z490.1-2016, Section 3.4.2, recommends evaluating three important elements of a safety training program.

• How the training program is managed
• The quality of the training process
• The results of training

Let's look at each of these three important elements more closely.

Evaluating the Training Program

Training works best when it's designed and performs as part of an integrated safety management system rather than a series of unrelated training sessions. Below are elements that should be included in the evaluation of a training program.

• Responsibility and accountability
• Quality of facilities and equipment
• Adequacy of program development
• Quality of delivery
• Completeness of documentation and records
• Adequacy of the safety training program evaluation process

Evaluating the Training Process

In an effective training program the training should be conducted using a systematic process that includes a needs assessment, objectives, course materials, lesson plans, evaluation strategies, and criteria for successful completion. Areas of emphasis include:

• training goals;
• learning environment;
• learning objectives; and
• training effectiveness.

Evaluating the Training Results

By evaluating the results of training, it's possible to make improvements to existing plans and gain awareness of the need for new training. Items to evaluate include:

• quality of the training action-plan;
• adequacy of long-term planning;
• use of needs assessment;
• appropriateness of prioritizing training; and
• adequacy of support and funding.

Instructions

Before beginning this quiz, we highly recommend you review the module material. This quiz is designed to allow you to self-check your comprehension of the module content, but only focuses on key concepts and ideas.

Read each question carefully. Select the best answer, even if more than one answer seems possible. When done, click on the "Get Quiz Answers" button. If you do not answer all the questions, you will receive an error message.

Good luck!