The scenario: Your safety and health program is in place. You have set your goal for the year and clearly stated the objectives, procedures, and activities necessary to meet that goal. Responsibilities have been defined and clearly assigned. Adequate authority and resources have been allocated. People have been trained in their safety and health program roles, and they understand the consequences of failing to perform their assignments.
Your responsibility for employee safety and health does not stop here. The next step -- a critical one -- is to evaluate how well your safety and health program is working.
This process is more than an inspection or an audit. Inspections are necessary to look at the facility, the process, and the individual jobs in order to identify and then to eliminate or control any hazards that may exist. Audits focus on program activities and seek to determine whether specific objectives have been met. For example, if you are assessing employee participation by looking at the activities of the safety committee, you will want to know if that committee met at the intervals specified, and if most of the members attended each meeting. These are audit questions.
But beyond this simple accounting are larger questions. For example, has employee participation at safety committee meetings helped improve the worksite's safety and health program? How is the work of the safety committee helping you meet your goal? These are the kinds of issues addressed by an evaluation.
A safety and health evaluation looks at the systems you have created to carry out your safety and health program. It asks if these systems are working effectively and efficiently. All systems that contribute to your safety and health program should be reviewed. These should include management leadership and the evaluation of that leadership, the analysis of the worksite to identify hazards, hazard prevention and control, accident and near-miss investigations, employee involvement, safety and health training, use of personal protective equipment (PPE), the health program, and the emergency response program. Your site may have additional programs or systems that contribute to the safety and health program. These also will need evaluating.
Who should conduct the evaluation? Although evaluations can be performed by worksite employees, they are best done by people who are knowledgeable about the site's processes and about managing safety and health programs, but who do not work at the site being evaluated.
Evaluation often causes anxiety for workers. You may be able to reduce that anxiety by letting your employees know that the evaluator will be focusing on systems and not on people.
Three useful tools for this evaluation are document review, employee interviews, and review of site conditions. These tools will provide you with the basis for an evaluation report. This report should contain a list of the programs or systems reviewed and a narrative account of the examination of each system or program. It also should contain a schedule of needed changes, with target completion dates, responsible parties, and space to indicate the date when changes are actually completed. Some reports include photographs of excellent situations and those needing improvement. Some provide a grading system, so that each year's results can be compared quickly to previous years. This report should be available to any employee who wants to read it, so it should be written in a straightforward and understandable way, avoiding jargon.
This chapter will explain in detail how you can accomplish this evaluation.
Ideally, everything that you know to be contributing directly to your safety and health program should be evaluated. OSHA's Safety and Health Program Management Guidelines can help you determine which areas of your program need evaluation. These are the four major areas of the Guidelines, what OSHA calls elements:
Evaluators can be drawn from the workplace safety and health department or the safety committee, but the best evaluators will be people possessing fresh vision. They will not be involved in the day-to-day operations of the site. Look in the corporate safety department, another worksite of the company, insurance companies, and outside consulting firms. Or have two activity managers switch places and evaluate each other's results.
Many workers' compensation insurance carriers offer their clients the services of a certified safety professional and a certified industrial hygienist. These experts are qualified to review your program activities.
Evaluators should be knowledgeable in the technical aspects of occupational safety and health, the management of safety and health, and the evaluation of programs. Of these three areas, management of safety and health is the most important. One source of assistance for training evaluators in safety and health management is the VPP Participants' Association, P.O. Box 991, McLean, VA 22101, telephone (703) 761-1146.
OSHA has found that there are three indispensable evaluation tools for judging the effectiveness of occupational safety and health program management. These are:
Documentation
Every worksite will have, at an absolute minimum, written accident reports and the OSHA log of injuries and illnesses as required by law. Major companies should have written procedures and records of all their safety and health programs. The evaluator should compare the written program to the written records of what occurred.
Interviews
In addition to the documentation, interviews can be very helpful in establishing what has occurred. OSHA uses two kinds of interviews, called formal and informal. The formal interviews are conducted privately with randomly selected employees who are asked preselected questions. Informal interviews occur at employee work stations and generally follow a list of topics.
To assess how well the worksite safety and health policy is communicated and understood, and how well the disciplinary system is working, ask the employees to explain them.
To gauge the effectiveness of safety and health training, interview hourly employees and first-line supervisors. Ask employees to describe what hazards they are exposed to, and how they are protected. Ask them to explain what they are supposed to do in several different types of emergencies. Ask supervisors how they teach, how they reinforce the teaching, how they enforce safety and health rules and safe work practices, and what their responsibilities are during emergency situations.
Interviews with management should focus on its involvement in and commitment to the safety and health program. Ask how the policy statement was created, and how that statement is communicated to all employees. Ask what information management receives about the safety and health activities, and what action management takes as a result of that information. Ask how management's commitment to safety and health is demonstrated to the workforce.
Review of Site Conditions
The conditions at the worksite reveal much about the safety and health program's effectiveness. Worksite conditions can be observed indirectly by examining documents such as inspection reports of hazards, employee reports of hazards, and accident/incident investigations.
Site tours also may reveal hazards. Be careful, however, that the site tour does not become a routine inspection, with emphasis only on hazard correction. When a hazard is found, certainly take steps to ensure its correction. But in addition, ask what management system(s) should have prevented or controlled the hazard. Determine why system(s) failed, and either change them or take other appropriate corrective measures.
Time and resources can be wasted when safety and health program activities do not achieve the desired results. Each year, activities should be planned with the intention of achieving specific objectives. These objectives, in turn, are geared toward reaching the safety and health program's goal.
As an example, a company's goal is:
To reach this goal, one objective this year is:
The company expects that achieving this objective will require more than 1 year. For the current year, the company plans to undertake two activities, each with governing procedures:
Activity 1: Create preventive maintenance checklists for all classes of company vehicles. Procedure: By Feb. 1, Transportation Department Chief will hold joint meeting of all drivers and vehicle maintenance mechanics to determine maintenance needs and create checklist of preventive maintenance tasks. Checklist will assign responsibilities to appropriate staff, indicate required time frames, and provide for sign-off.
Activity 2: Conduct a survey of non-vehicle machinery throughout the worksite to determine preventive maintenance needs. Procedure: By Feb. 1, each Department will submit to Maintenance Department a list of all machinery located within the Department, together with notations about operating problems, hazards, maintenance needs. By Mar. 1, Maintenance Department Chief and staff will visit each Department to examine machinery and to discuss needs with operators. By Apr. 1, a comprehensive report will be submitted to Vice-President for Operations, inventorying machinery and indicating maintenance needs and suggested maintenance schedule.
The end-of-year safety and health program evaluation will determine whether these activities were conducted and whether they had the desired effect, i.e., successfully began the process of developing a comprehensive preventive maintenance program. The evaluation will then examine the value of this objective: did the achievement of this phase of a preventive maintenance program move the company closer to its targeted safety and health goal? If this analysis finds program efforts that are ineffective and do not contribute to the goal, the evaluation should include recommendations for program changes or deletions or additions for the next year.
Activities and Procedures
Do the actual safety and health program activities and the procedures for implementing them bring the expected results?
Larger worksites. Large companies will have written procedures for the major activities of their safety and health program. They also will have written records of those activities as they were performed. Evaluating whether the written procedures were followed in the period evaluated, or how well they were followed, is an audit function of quality assurance or program evaluation.
Smaller worksites. Even if yours is a smaller business with more limited recordkeeping, you still should put some effort into thinking about how safety and health activities were carried out for the period evaluated, and whether the results achieved were those expected at the outset.
Sample questions. The precise questions you should ask will depend upon the activity being audited and the way the activity was to be accomplished. For example, if plans call for a certain person to carry out the inspection program, did this person actually conduct the inspections? In many workplaces, inspections are conducted by the person with the most expertise along with members of the site safety and health committee. Was the expert present during every inspection? Did the employee members always participate?
Other questions about inspections might include the following:
Similar questions should be asked about each activity under the safety and health program. When a discrepancy is found between the original plan and actual execution of the activity, assess which way best meets the safety and health objectives and goal. Then make sure that everyone follows that procedure.
Objectives
The objectives connect the goal for the safety and health program to the program procedures and activities.
Objectives that can be audited. Sometimes a program objective will be to complete a new or improved activity. For example, suppose the objective states, "Complete one job safety analysis each month, with follow-up revision of safe work procedures and employee training in the following month." In this case, the objective describes the frequency of activities rather than the desired result. An evaluation of this objective involves no more than determining whether the activities occurred. Therefore, an audit will be appropriate. Look for evidence that job safety analyses were done each month. Is there evidence that revisions of procedures and training also were made each month as a result of the previous month's job safety analysis? If the answer is yes, and other program evaluation reveals no need to do anything differently, the frequency of these activities will become an ongoing subject of audit.
Objectives that must be evaluated. Ordinarily, objectives should focus on the results desired from the program activities. For example, an objective might state, "Identify and assign all areas of safety and health responsibility that are not presently clearly assigned, so that all safety and health responsibilities can be successfully carried out." A set of activities will be needed to accomplish this objective. The activities might include assigning a committee to list all the safety and health responsibilities; reviewing assignment of those responsibilities; identifying missing, duplicate, and unclear assignments; and recommending clearer assignments.
In this case the evaluation will focus on whether the objective was accomplished. That is, were all areas of safety and health responsibilities that were not clearly defined actually identified and clarified? Perhaps these areas were identified but the corrections not made because the Personnel Department needed to first rewrite seven major job descriptions to get the responsibilities correctly assigned. This, then, leads to a new objective, to get the seven job descriptions rewritten with clearer assignment of safety and health responsibilities. If all safety and health areas have been identified and clarified, this objective will not lead to another objective.
As another example, say the objective is to increase employee safety and health awareness by involving the employees' families in a safety and health awareness program. The evaluation will seek to measure the difference between employee awareness before the family program and after it. One way to measure that difference is to ask the employees who participated in the program to complete a questionnaire. Another is to select employees randomly and ask them if they thought their awareness was increased. Still another is to ask supervisors if they perceived a difference in employee awareness after the family program began. If, after using some type of meaningful measurement tool, the findings indicate that the program has increased awareness, it will make sense to designate the family program an ongoing activity subject to audit. If the findings show no measurable increase in awareness, then the family program can be changed or another activity substituted. The altered objective will then be to increase employee awareness by this new means.
Goal
The goal is the ultimate intention of the safety and health program, its basic aim.
Objectives should be evaluated to make sure they are leading to the program goal. For example, suppose the goal is the development of a comprehensive and effective safety and health program, and one of the objectives intended to achieve that goal is, "Hold monthly safety meetings for all employees." Since this is an activity objective, it can be audited to determine whether safety meetings are actually being held. But the next question is, "Did achieving this objective help fulfill the goal of the safety and health program?" In other words, did the safety meetings help employees understand the hazards to which they are exposed and result in plans to reduce exposure?
There are various ways to collect such information. A sign-off sheet can indicate who attended the training. The same sheet can ask employees to describe hazards and potential hazards that exist in their work area and ways to better control them. If there is no sign-off sheet, interviews with some randomly selected employees can reveal their opinion of whether the meetings improved their understanding of hazards and resulted in plans to control them. Interviews with supervisors can reveal whether employees exhibited better understanding after training.
If the results indicate that not much was learned at these sessions, ask further questions to see what went wrong. In this way, all the objectives can and should be checked to see if they are helping achieve the goal.
Program evaluation can identify activities that are not really helping to improve worksite safety and health. In so doing, evaluation can save you time, effort, and money.
Evaluation Judgments
The important work of gathering information about safety and health program activities is the most time-consuming part of program evaluation. It is, however, the easiest to understand and accomplish. The hardest part is making judgments about program effectiveness.
Employers should draw up appropriate, site-specific procedures for gathering information and making judgments. You may also want to add environmental, product safety, or security considerations to the evaluation process. The questions you ask should be based on individual site program activities and site objectives.
Insist that the evaluator determine the program's bottom line profitability, its real benefit. In other words, which activities contribute to the safety and health goal, and which do not? Judgments and decisions made by evaluators should be driven by this quest for profitability, by which we mean improvement in safety and health protection. Do not accept a narrative that only describes the program. Insist that the hard questions about program effectiveness be addressed.
The evaluation will prove valuable only if it leads to improved performance in meeting the safety and health goal. Some of the recommendations that result from the evaluation will be for one-time corrections. Many, however, will involve changing emphasis or trying new activities. These recommendations should be incorporated into the objectives for the next year. Consider establishing, as a permanent objective, an audit of the procedures that your program sets for safety and health program activities.
Larger Worksites
The evaluation should result in a written report with written recommendations and documented follow-up to those recommendations. It may be useful to refer to past years' evaluations when preparing new ones or when planning new objectives. If you find that the same recommendations are being made year after year, the process of implementing and tracking recommendations to completion needs improvement.
Smaller Worksites
At smaller sites a written evaluation report may not be practical. However, it is important to set aside time to think about desired changes. The evaluation process already has involved considering what was done during the course of the year, talking to people, looking at the site's working conditions, and reviewing available documents. Next, decide what you want to do differently and make sure that everyone understands what is expected.
This chapter has defined a safety and health program evaluation. It has described what should be evaluated, who should do the evaluation and with what tools, how the evaluation should be conducted, and how to use the results.
By using this information to perform annual evaluations, you will be able to compute your company's safety and health bottom line, just as you now can calculate your organization's financial bottom line. You will have the information needed to make knowledgeable and effective decisions promoting workplace safety and health.
Source: Missouri Department of Labor and Industrial Relations
Copyright ©2000-2019 Geigle Safety Group, Inc. All rights reserved. Federal copyright prohibits unauthorized reproduction by any means without permission. Disclaimer: This material is for training purposes only to inform the reader of occupational safety and health best practices and general compliance requirement and is not a substitute for provisions of the OSH Act of 1970 or any governmental regulatory agency. CertiSafety is a division of Geigle Safety Group, Inc., and is not connected or affiliated with the U.S. Department of Labor (DOL), or the Occupational Safety and Health Administration (OSHA).
