Measuring Safety and Health Performance – Part 1

Part 1: A review of commonly used performance indicators.

By Jerome E. Spear

How an organization or business measures performance undoubtedly influences its behavior. But measuring safety is difficult because it is hard to predict the impact new safety metrics will have on individual behavior, attitudes, and the overall safety climate.

Regardless how difficult it is to measure, a firm’s safety performance needs to be measured. Without measurement, accountability becomes meaningless.

But before various measurement systems are devised, their purpose and limitations need to be thoroughly understood. This two-part series, the second half of which will appear in the February issue of Water Well Journal, describes different types of safety performance measures and their benefits and limitations.

Metric Categories

There are many ways to classify safety performance measures—with trailing or leading indicators, outcome or process oriented, results or activity-based measures, downstream or upstream factors, and qualitative or quantitative metrics.

For the sake of simplicity, the safety metrics described in this series are classified as outcome or process oriented.

Outcome performance measures are “after the fact.” The activity occurs and the outcome is measured. Two common examples of outcome metrics are injury/illness rates and workers’ compensation claims data.

A firm’s safety performance needs to be measured. Without measurement, accountability becomes meaningless.

Process-oriented performance measures, on the other hand, are those measures indicating an action or activities performed. Ideally, process measures should be linked—and statistically validated—to outcome metrics, but this is rarely done. As a result, process metrics do not necessarily predict a program’s outcome, but rather, indicate the extent to which an activity or process has been implemented (Janicak 2003).

In general, there is no single reliable measure of safety and health performance. Instead, a mixture of both outcomeoriented and process-oriented measures are needed to effectively evaluate performance. Furthermore, the types of metrics used should be different for evaluating different levels of the organization or business.

Safety pioneer Dan Petersen suggested that only processoriented metrics be used at the lower managerial levels and activity measures (with some outcome measures) primarily used for the middle-upper management levels. Pure outcome measures should be reserved for the executive level (Petersen 1996). He added ideally the metrics should be integrated and linked to the overall vision, goals, and objectives of the company.

Outcome-Oriented Measures

Injury/Illness Rates

While occupational safety and health performance is often measured with injury/illness rates and workers’ compensation claims data, the most common outcome metrics are based on injury/illness rates such as the OSHA recordable incident rate.

The OSHA recordable incident rate is an outcome metric measuring the number of employees per 100 workers who have sustained an OSHA recordable injury or illness. The benefit of using injury/illness statistics as a safety performance metric are they are easy to use and injury/illness data are readily available.

Plus, OSHA recordable rates by SIC codes (Standard Industrial Classification) are published annually by the Bureau of Labor Statistics. This allows companies to compare their injury/illness rates to their respective industry average.

There are many limitations, though, with using injury/illness rates as a primary performance indicator.

  1. Injury/illness rates are inherently linked to bad news. Emphasizing these rates typically rewards employees too much for not reporting injuries or illnesses, thus preventing the root causes of problems to be properly investigated and corrected. This is potentially the most significant limitation.
  2. There can be considerable variations in interpreting and applying OSHA recordkeeping guidelines. Therefore, generalizing the OSHA log information from one company to another is questionable because of the various reporting techniques between workplaces.
  3. OSHA rates, to a large extent, depend on the medical treatment given to the injured employee. OSHA rates can be influenced by how a company manages the medical treatment received. The medical community is inconsistent in treating injuries.
  4. Injury/illness rates provide little feedback for safety improvement. It is not authoritative in nature, thus provides little or no information on how a company can improve.
  5. Injury/illness rates often do not reflect the potential severity of an event, merely the consequence. Whether a particular event results in an injury is often a matter of chance. It will not necessarily reflect whether or not a hazard is under control.
  6. A low injury rate can lead to complacency. Likewise, having a single OSHA recordable injury might cause management to overreact.
  7. There must be an injury or illness in order to get a data point. Injury statistics reflect outcomes, not causes. A low injury rate results in few data points being available.
  8. Smaller companies with fewer man-hours are more susceptible to wide fluctuations in injury rates. OSHA recordable incidents account for a small number of workplace accidents. The causes of such incidents may not represent the norm.

Most safety professionals agree with the many pitfalls of using these traditional outcome metrics as the primary performance indicator.

However, many customers and upper management object to the removal of such outcome metrics. OSHA also requires recordable injuries be logged and tracked. Therefore, it is unlikely these traditional performance metrics will ever be eliminated.

The key is to develop additional, more useful, performance indicators in an effort to shift the focus away from the traditional, less useful, outcome metrics.

Insurance Claims Data

Financial calculations based on insurance claims data are other popular sources for outcome measures. The data include the experience modification rate, loss ratio, and total losses incurred.

The experience modification rate (EMR) is a factor applied to the manual premium to either increase or decrease the insured’s final premium. The rate is determined by the Council on Compensation Insurance or other insurance rating groups.

DACUM CODES
To help meet your professional needs, this column covers skills and competencies found in DACUM charts for drillers and pump installers. DO refers to the drilling chart and PI represents the pumps chart. The letter and number immediately following is the skill on the chart covered by the column. This column covers: DOC-2, DOD-8, DOK-4, DOK-8, DOK-9, PIB-2, PIG-1, PIG-3

More information on DACUM and the charts are available at www.NGWA.org.

The EMR calculation is based on the type of work performed and past loss history. The average for an industry is represented by an EMR of 1.0. Companies with EMRs greater than 1.0 are paying a workers’ compensation premium greater than the industry average.

Companies with an EMR less than 1.0 are paying premiums less than the industry average. The loss ratio is a formula used by insurers to relate loss expenses to income and is closely related to the EMR.

The loss ratio is calculated by the following formula:

Incurred losses + Adjustment expenses
____________________________
Earned premium

If the loss ratio is greater than 1, or 100%, the insurance carrier is paying more to provide coverage for the insured than it received in premiums.

The primary benefit of using the EMR as a performance indicator is it is directly related to operational costs. The rate has also already been normalized for company size, payroll data, nature of operations, and other factors. Therefore, no other data is needed for comparison.

However, there are some limitations in using EMR as a sole performance indicator. The rate is averaged over a significant period of time. Thus an employer who has improved safety performance may still suffer the impact of previous years of poor performance, while an employer with a good EMR who has let safety decline will have a lower EMR until losses enter the formula.

Additionally, when a claim occurs, the insurer establishes a reserve, which may equal the maximum probable loss from that injury. However, claimsreserving practices differ among insurance carriers, which is another limitation of the EMR or other measures based on insurance claims data.

Look for part two of the series in the February issue, which will examine process-oriented methodology for measuring safety performance, such as safety audits and safety training.

References

Birkner, L., and R. Birkner. 1999. Determining the value of occupational hygiene and safety. Occupational Hazards 61, no. 2: 51-52.

Birkner, L., and R. Birkner. 1999. The health and safety scorecard. Occupational Hazards 61, no. 6: 51-52.

Furst, P.G. 2006. Measuring success—Integrated risk management. Retrieved on web December 16, 2008 from www.irmi.com/Expert/Articles/2006/Furst06.aspx.

Janicak, C. 2003. Safety Metrics: Tools and Techniques for Measuring Safety Performance. Lanham, Maryland: Government Institutes.

Kaplan, R.S., and D.P. Norton. 1996. Using the balanced scorecard as a strategic management system: Building a scorecard can help managers link today’s actions with tomorrow’s goals. Harvard Business Review 1: 75-85.

Petersen, D. 1996. Safety by Objectives: What Gets Measured and Rewarded Gets Done 2nd Edition. New York, NY: John Wiley & Sons Inc.

Petersen, D. 1998. What measures should we use and why? Professional Safety 43: 37-40.

Phillips, J. 1991. Handbook of Training Evaluation and Measurement Methods. Houston, Texas: Gulf Publishing Company.

Veley, C., N. Richie, A. Coats, J. Disatell, and P. Cook. 2004. A new method of measuring safety performance will soon affect the whole industry. Paper presented at the Seventh Society of Petroleum Engineers International Conference on Health, Safety, and Environmental in Oil and Gas Exploration and Production. SPE: Richardson, Texas


Jerome E. Spear, CSP, CIH, is president of J.E. Spear Consulting and has more than 22 years of experience helping organizations prevent injuries and illnesses, control losses, and achieve regulatory compliance. He held the positions of technical services manager with XL Specialty Risk Consulting and corporate industrial hygiene manager for Chicago Bridge and Iron Co., a worldwide steel fabricator and construction company.

Be the first to comment

Leave a Reply

Your email address will not be published.


*