To make sure that the training program is accomplishing its goals, an evaluation of the training can be valuable. Training should have, as one of its critical components, a method of evaluating the student's RKSAs:
The evaluation also include an analysis of the quality of the training process and impact of training on the organization.
It's important that a written plan for evaluating the training be developed when the course objectives and content are developed. It should not be delayed until the training has been completed.
Evaluation will help determine the amount of learning achieved by training and whether an employee’s performance has improved on the job as result of training. Among the methods of evaluating training are:
However it is conducted, an evaluation of training can give employers the information necessary to decide whether the employees achieved the desired results, and whether the training session should be offered again at some future date.
As outlined in detail in ANSI Z490.1, Section 3.4, the employer should design evaluation strategies to measure the performance of the safety training program. To do this, one or more evaluation techniques may be used to evaluate training. The evaluation techniques mentioned in the ANSI standard mirror those developed by Donald Kirkpatrick, author of Evaluating Training Programs: The Four Levels. As the title of the book implies, he developed a four-level model for evaluating training. The first three levels of evaluation, those that you'll be involved with most of the time, measure the impact of training on the leaner. The fourth level measures the impact on the organization. Let's take a look at these four levels of evaluation.
Level One evaluation measures the performance of the training event by gathering data from students and others about the quality of the training process. We're interested in learning what the trainee, or other evaluator, thought about the content and trainer's presentation of the training. Learner evaluation of training typically occurs at the end of training, but may also be conducted during the training session or at some time after the completion of training. The typical tool to measure reaction is a survey.
Level Two evaluation measures what the learner knows and can do after training. The vast majority of your safety training will require this level of evaluation. What the learner knows or can do can be measured during and at the end of training as long as it's in the learning environment that does not exposure the learner to actual hazards. Trainer evaluation assesses the extent to which students have advanced in skills, knowledge, or abilities (SKAs). Evaluation at this level is suitable for certifying employees as "initially qualified." Methods to evaluate SKAs at this level include:
On-the-job training (OJT) is a very effective training strategy to test SKAs. No matter the training strategy used, be sure evaluation measures are reliable and valid. They are reliable if the results are consistent. They are valid if the results reflect the SKAs specified in the learning objectives.
Test Out option. Evaluation takes place while the learner is in the training environment. Additionally, it may be appropriate, in some instances, to allow learners to "test out" by demonstrating the ability to achieve course objectives without actually being required to complete training. If you allow this option, make sure learners understand test-out criteria, and be careful to ensure training complies with government regulations.
Level Three evaluation is interested in measuring success the learner has had in applying acquired knowledge and skill to their job. It answers the question - Are the newly acquired skills, knowledge, or abilities being successfully used in the workplace? Observation of performance is the technique used for this level of evaluation. Evaluation takes place at some time (days, weeks, months) after the learner leaves the training environment. Typically, a trainer or supervisor will observe the employee at work and rate his or her performance against learning objectives. Certification at this level may be used to certify an employee "fully qualified." It's important to include Level 3 evaluation because it:
According to the Four-Level Evaluation model, techniques (tests) to evaluate should always include level one techniques and move sequentially through levels two, three, and four. Information from each prior level serves as a base for the next level's evaluation.
Level Four evaluation represents a change of focus. At this level, we're interested in the degree to which training has effectively contributed to the overall success of the company rather than the learner. At this level, the performance of employees who have received training is usually contrasted with the performance of a control group. Although Level Four evaluation is not required by OSHA standards, it's still considered quite important and should be thought of as a wise business practice. We're no longer measuring the learner or the presentation, we're evaluating the impact of safety training on a number of results, including:
ANSI Z490.1 recommends evaluating three important elements of a safety training program.
Let's look at each of these three important elements more closely.
Training program management. Training works best when it's designed and performs as part of an integrated system management system rather than a series of unrelated training sessions. Elements to evaluate include:
The training process. Training should be conducted using a systematic process that includes a needs assessment, objectives, course materials, lesson plans, evaluation strategies, and criteria for successful completion. Areas of emphasis include:
Training results. By evaluating the results of training, it's possible to make improvements to existing plans and gain awareness of the need for new training. Items to evaluate include:
OSHA's Occupational Safety and Health Standards, CFR 29, Subpart: H - Hazardous Materials 1910.120 App E Training Curriculum Guidelines, serves as an effective guideline for evaluating all safety training. According to these guidelines:
Factors listed here are suggested criteria for determining the quality and appropriateness of employee health and safety training.
A. Training Plan. Adequacy and appropriateness of the training program's curriculum development, instructor training, distribution of course materials, and direct student training should be considered, including:
B. Program management, Training Director, staff, and consultants. Adequacy and appropriateness of staff performance and delivering an effective training program should be considered, including:
C. Training facilities and resources. Adequacy and appropriateness of the facilities and resources for supporting the training program should be considered, including:
D. Quality control and evaluation. Adequacy and appropriateness of quality control and evaluation plans for training programs should be considered, including:
E. Students. Adequacy and appropriateness of the program for accepting students should be considered, including:
F. Institutional Environment and Administrative Support. The adequacy and appropriateness of the institutional environment and administrative support system for the training program should be considered, including:
G. Summary of Evaluation Questions. Key questions for evaluating the quality and appropriateness of an overall training program should include the following:
Source: OSHAcademyCertisafety Section Home Page
Copyright ©2000-2016 Geigle Safety Group, Inc. All rights reserved. Federal copyright prohibits unauthorized reproduction by any means without permission. Students may reproduce materials for personal study. Disclaimer: This material is for training purposes only to inform the reader of occupational safety and health best practices and general compliance requirement and is not a substitute for provisions of the OSH Act of 1970 or any governmental regulatory agency. CertiSafety is a division of Geigle Safety Group, Inc., and is not connected or affiliated with the U.S. Department of Labor (DOL), or the Occupational Safety and Health Administration (OSHA).