HOME |  COURSES   HELPS   AUDITS   STORE   SERMONS   NOTES   NEWS   CERTIFICATION   CSHM   BOOKS   LINKS   PRIVACY   CONTACT
Free and Low Cost Online Safety Training and Course Materials for Occupational Safety and Health Professionals

ANSI Z490.1-2001 Training Evaluation Guidelines

As outlined in detail in ANSI Z490.1, Section 3.4, the employer should design evaluation strategies to measure the performance of the safety training program. To do this, one or more evaluation techniques may be used to evaluate training. The evaluation techniques mentioned in the ANSI standard mirror those developed by Donald Kirkpatrick, author of Evaluating Training Programs: The Four Levels. As the title of the book implies, he developed a four-level model for evaluating training. The first three levels of evaluation, those that you'll be involved with most of the time, measure the impact of training on the leaner. The fourth level measures the impact on the organization. Let's take a look at these four levels of evaluation.

Evaluate Reaction, Knoweldge, Skills, Application, and Impact (RKSAI)

To make sure the training program is conforming to ANSI/ASSE Z490.1-2001, an evaluation strategy to measure results of training on the employee and the organization is required. Training should include methods to evaluate :

  • employee reaction (R) to training
  • knowledge (K) and skills (S) gained during the training
  • application (A) of the knowledge and skills in the workplace
  • the impact (I) of training on the organization
It's important that a written plan for evaluating the training be developed when the course objectives and content are developed. It should not be delayed until the training has been completed.

Purpose of Training Evaluation

Evaluation will help determine the amount of learning achieved by training and whether an employee’s performance has improved on the job as result of training. Among the methods of evaluating training are:
  1. Employee opinion. Questionnaires or informal discussions with employees can help employers determine the relevance and appropriateness of the training program.
  2. Supervisor observations. Supervisors are in good positions to observe an employee’s performance both before and after the training and note improvements or changes.
  3. Workplace improvements. The ultimate success of a training program may be changes throughout the workplace that result in reduced injury or accident rates.

However it is conducted, an evaluation of training can give employers the information necessary to decide whether the employees achieved the desired results, and whether the training session should be offered again at some future date.

The Four Levels of Evaluation

Level One Evaluation: Measures Reaction (R)

Level One evaluation measures the performance of the training event by gathering data from students and others about the quality of the training process. We're interested in learning what the trainee, or other evaluator, thought about the content and trainer's presentation of the training. Learner evaluation of training typically occurs at the end of training, but may also be conducted during the training session or at some time after the completion of training. The typical tool to measure reaction is a survey.

Level Two Evaluation: Measures knowledge (K), skills (S)

Level Two evaluation measures what the learner knows and can do after training. The vast majority of your safety training will require this level of evaluation. What the learner knows or can do can be measured during and at the end of training as long as it's in the learning environment that does not exposure the learner to actual hazards. Trainer evaluation assesses the extent to which students have advanced in knowledge and skills. Evaluation at this level is suitable for certifying employees as "initially qualified." Methods to evaluate knowledge and skills at this level include:
  • written or verbal pre/post training exams,
  • skill/performance demonstrations

On-the-job training (OJT) is a very effective training strategy to test knowledge and skills. No matter the training strategy used, be sure evaluation measures are reliable and valid. They are reliable if the results are consistent. They are valid if the results reflect the knowledge and skills specified in the learning objectives.

Test Out option. Evaluation takes place while the learner is in the training environment. Additionally, it may be appropriate, in some instances, to allow learners to "test out" by demonstrating the ability to achieve course objectives without actually being required to complete training. If you allow this option, make sure learners understand test-out criteria, and be careful to ensure training complies with government regulations.

Level Three Evaluation: Measures application (A) of Knowledge and Skills

Level Three evaluation is interested in measuring success learners demonstrate in applying acquired knowledge and skills to their job. It answers the question - Are the newly acquired knowledge and skills being successfully used in the workplace? Observation of performance is the primary technique used for this level of evaluation. Evaluation takes place at some time (days, weeks, months) after the learner leaves the training environment. Typically, a trainer or supervisor will observe the employee at work and rate his or her performance against learning objectives. Certification at this level may be used to verify an employee "fully qualified." It's important to include Level 3 evaluation because it:
  • Measures long-term retention of knowledge. When the evaluator asks the employee to explain procedures at some point in time after training, we can determine how well the employee has retained necessary knowledge.

  • Measures skills after training. Through observation, the evaluator can determine the degree to which the employee has retained the skills necessary to perform the procedure.

  • Validates the safety training. Successful performance in the work environment is a solid indicator that the safety training received by the employee was adequate.

  • Validates the safety culture. Successful performance at work generally indicates the safety management system and culture is supporting the safety training program.

  • It's efficient. Supervisors can perform this level of evaluation during the normal course of their daily supervision. No special procedure is required.

According to the Four-Level Evaluation model, techniques (tests) to evaluate should always include level one techniques and move sequentially through levels two, three, and four. Information from each prior level serves as a base for the next level's evaluation.

Level Four Training - Measures the Impact (I) of Training

Level Four evaluation represents a change of focus. Now we're interested in the degree to which training has effectively contributed to the overall success of the company. The performance of employees who have received training is usually contrasted with the performance of a control group that has not had the training. Behaviors and results may also be evaluated before and after training.

Although Level Four evaluation is not required by OSHA standards, it is required by ANSI/ASSE Z490.1-2001. We're no longer measuring the learner or the presentation, we're evaluating the impact of safety training on a number of results, including:

  • compliance behaviors
  • hazard identification and control skills
  • frequency of non-injury incidents
  • workers compensation claim costs
  • direct/indirect accident costs
  • production costs
  • turnover and absenteeism
  • frequency of grievances
  • profitability
  • morale

ANSI/ASSE Z490.1 Guidelines for evaluating training programs, process, and results

ANSI/ASSE Z490.1-2001, Section 3.4.2, recommends evaluating three important elements of a safety training program.

  • How the training program is managed
  • The quality of the training process
  • The results of training

Let's look at each of these three important elements more closely.

Training program management. Training works best when it's designed and performs as part of an integrated system management system rather than a series of unrelated training sessions. Elements to evaluate include:
  • Responsibility and accountability
  • Quality of facilities and equipment
  • Adequacy of program development
  • Quality of delivery
  • Completeness of documentation and records
  • Adequacy of the safety training program evaluation process

The training process. Training should be conducted using a systematic process that includes a needs assessment, objectives, course materials, lesson plans, evaluation strategies, and criteria for successful completion. Areas of emphasis include:
  • Training goals
  • Learning environment
  • Learning objectives
  • Training effectiveness

Training results. By evaluating the results of training, it's possible to make improvements to existing plans and gain awareness of the need for new training. Items to evaluate include:
  • Quality of the training action-plan
  • Adequacy of long-term planning
  • Use of needs assessment
  • Appropriateness of prioritizing training
  • Adequacy of support and funding

Source: OTN, Steven Geigle, M.A., CSHM

HOME |  COURSES |  TRAINER HELPS |  AUDITS |  SERMONS |  NOTES |  CERTIFICATION |  CSHM |  BOOKS |  PRIVACY |  CONTACT
Copyright © 2000-2006 Geigle Communications. All rights reserved. Federal copyright law prohibits unauthorized reproduction by any means and imposes fines up to $25,000 for violations.