Information needed for decision making is characterized by its precision and certainty. The level of precision and certainty is balanced by our willingness to expend resources to obtain it. Generally, highly precise, highly certain information is very expensive to obtain.

To make risk-based decisions, the decision maker must understand how future accidents can occur. For example, information on historical performance may be available, but the decision maker believes that this information does not adequately predict the existence of other potential accidents.

Dealing with information precision

The precision of information is characterized by its level of detail. For example, a person can be from the United States, Texas, or Dallas. Likewise, a number can be described by the number of places following the decimal point. The more precise the information, the more detail is inherent in it.

The decision maker must understand the precision required to make a decision. If knowing that a person is from Texas is sufficient to make a decision, there is no need to expend resources to determine which city in Texas the person is from. Likewise, if numerical information to one decimal place is precise enough to make a decision, information precise to three decimal places will not affect it.

Dealing with information uncertainty

In any decision-making process, there is constant struggle between the need for more and better information and the practicality of improving the information. This is illustrated by the simple figure below.

Even when a lot of information is collected, a great deal of uncertainty remains. So the decision makers and information suppliers must work together to make sure that the cost of collecting more accurate data does not outweigh the benefits of having it. This is why analysts should never use very complex risk assessment tools without first trying to meet decision-making needs with simpler tools.

Dealing with uncertainty is part of any decision-making process. Therefore, those taking part in decision making, either directly or indirectly, must be aware of the most likely sources of uncertainty: model uncertainty and data uncertainty.

Model uncertainty

The models used in both the general decision-making structure and in detailed risk assessments will never be perfect. The detail in a model and scope boundaries will determine how well the model reflects reality. Even if the data are perfect, the model usually brings some doubt into the results.

For example, Phil was asked to describe what factors would influence his choice of a new car. If Phil cannot describe all of the factors that influence his choice, then these factors will not appear in his decision model.

More detailed levels of risk analysis can reduce model uncertainty by more thoroughly accounting for potentially important loss sequences. However, more thorough analysis also costs more.

The simplest risk assessments are historical event summaries and account only for known accidents, and possibly some near misses, that have occurred during some reporting period. Streamlined risk assessments require more resources, but they also account for more near misses, as well as other recognized accident scenarios that did not occur. More detailed risk assessments require even more resources, but they systematically identify and account for previously unrecognized accident scenarios.

Data uncertainty

Data uncertainty causes much concern during decision making. Data uncertainty arises from any or all of the following:

• The needed data do not exist
• The analysts do not know where to collect the data, or they do not have the staff, funds, or time to collect it
• The quality of the data is questionable, usually because of the methods used to gather it
• The data vary widely, making their use complex

Although steps can be taken to reduce uncertainty in data, all data have some uncertainty. This uncertainty cannot be ignored. Following are methods available for dealing with data uncertainty:

Subjectively characterize uncertainty (for example, as high or low). A simple approach in which doubt in the final answer is estimated based on personal experience or belief.

Perform calculations using best-case and worst-case situations. An approach that uses different calculations for best-case and worst-case conditions to reflect the range of possible outcomes.

Analyze a number of possible situations (i.e., what-if scenarios). An expanded version of the previous approach that involves calculations for many other sets of conditions, usually including an estimate of how likely each set is to occur.

Decrease the precision requirements. Using broader ranges when categorizing the frequency and consequence of accidents increases the certainty in the selection.

Perform calculations using probability distributions in place of discrete estimates. A more complicated approach that uses statistics to describe data used in a model so that statistical descriptions of the expected outcomes can be formed.

Choose a simple method first for dealing with uncertainty. If decision makers need better estimates, the uncertainty can be reduced for the issues that most affect the model.

Dealing with resource needs

The objective is to use the minimum resources necessary to develop the required information. One effective means of minimizing resources involves starting with the lowest-cost approach that can possibly provide needed information with the required precision and certainty. This strategy most often relies on "streamlined" forms of traditional risk assessment tools. For example, before requesting any detailed modeling, the decision maker might contact one or more system experts and simply ask their perception of the answer to the risk-based question. Based on their experience, the experts may be able to provide the needed results with adequate precision and certainty. The need for more detailed analysis is therefore avoided. Be ready to commission more detailed risk assessments, though, if results from the less detailed approaches are not suitable for making a decision.

Dealing with expert bias

Risk assessments often rely on expert judgments for estimates of frequencies, probabilities, and expected levels of consequence. Reliance on experts is necessary when loss data is nonexistent or when schedule/budget constraints do not allow collection and evaluation of loss data.

The down side of relying on expert judgment is that the expert(s) may be unintentionally (or intentionally) biased. Experts often rely on heuristics (rules of thumb) to reach judgments quickly. The following heuristics are known to cause experts to introduce bias in their responses:

• Availability - people judge the likelihood of an event by how easily they can recall or imagine examples.
  
• Representativeness - people judge an event by how closely it represents a stereotypical member of a group.
  
• Anchoring and adjustment - people often pick a starting value, or anchor, and then adjust up and down from it. They may not adjust adequately to a new value when new information comes to light
• Affect - positive information about the benefit of an activity can affect an individual's perception/inference of the risk associated with the activity, causing underestimation of the risk. Alternatively, negative information about the benefit of an activity can cause an individual to overestimate the risk.

The following are general guidelines to help reduce bias when querying experts.

• Encourage some anecdotal data (story-telling, etc.), but ask if reported events are truly representative of the norm, or are outliers.
  
• Ask the expert about the process used to arrive at a decision/conclusion.
  
• Reframe questions from different perspectives.
  
• Ask for upper and lower bounds before asking for the best estimate evaluation.
  
• Discuss the implications of selecting one answer versus another.
  
• Resolve apparent discrepancies or inconsistencies.
  
• Ensure results are plausible.

Barriers to Risk-based Decision Making

A common barrier to risk-based decision making is the perception that mounds of highly precise, technical data are required before a decision can be made. Overcome this perceived barrier by trying to develop the data from information that is already at hand. Even though the precision and certainty of this data may not be high, they may be high enough for the decision maker. When more detailed data are required, then you know that you have at least tried to develop the required decision-making information from what was immediately available to you using the minimum resources.

Another common barrier to risk-based decision making is the perception that the risk assessment part of the process takes far too much time. There is no question that more time is required for complicated decisions that use information developed from highly precise and certain data. However, risk-based decisions are often not this complicated. Do existing risk-based decision-making tools like checklists and risk indexes work? These tools take very little time, but they often end up providing the information needed to make the decision.

One impediment to risk-based decision making is found in the culture of "it's always been done this way." Challenge this thinking. Why has it always been done this way? Do regulations REQUIRE this decision to be made this way, or is this simply a convenient interpretation of a flexible rule?

Sometimes the prescriptive requirements that appear to be inflexible can be changed. Use the risk-based decision-making process to help change prescriptive requirements that do not effectively manage important risks.

Risk-based decision making is for everyone. An inexperienced person given basic training in the use of a well-developed risk-based checklist will make good risk-based decisions. Tear down barriers that cause people to believe risk-based decision making is only for the most experienced. Use the experienced people to help develop information for complex decisions and to create new risk-based decision-making tools. No one should perceive experience as a barrier to risk-based decision making.