Feature 1

Systematic review of feedback mechanism improving safety in dynamic and static industry

Muthammal Sadayappan and Dr. Farman A. Moayed, PE

Biographies

Muthammal Sadayappan

Muthammal Sadayappan – Student at Indiana State University majored in Human Resource Development at Masters level. Currently pursuing MBA at Indiana State University. Worked as an Executive Manager in S.V.S pharmaceutical company.

Dr. Farman A. Moayed

Dr. Farman A. Moayed – Assistant Professor in the Department of Health, Safety and Environmental Sciences at Indiana State University and a registered PE. He received his Ph.D. in Industrial Engineering from University of Cincinnati, OH (2008). Research interests are in safety management, lean manufacturing, total productive maintenance, production systems, and artificial neural networks.

 

Abstract

Background - Human resources is the most valuable and important resource. Behavior based safety (BBS) is one way the management can use to ensure safety. BBS aims at helping employees to identify and choose the safe behavior. BBS is based on the following four components: i. behavioral observation and feedback process, ii. formal review of observation data, iii. improvement goals; and iv. reinforcement for goal attainment and improvement.

Objective - This systematic review was aimed at studying the relationship between feedback mechanism and the reduced number of accidents, injuries and illness in dynamic and static industries.

Methods – Eight articles using an inclusion and exclusion criteria were chosen. Epidemiological Appraisal Instrument (EAI) was used to assess the quality of all the accepted papers in the following areas: study description, subject selection, observation quality, data analysis and generalization of results.

Results – The results showed that in both static and dynamic industries, feedback or combinations of feedback did reduce accidents, injuries and illness in workplace.

 

Full Text

Introduction

In 2008, the Bureau of Labor statistics reported preliminary count of 5, 071 fatal work injuries. In 2007, the recorded rate of non-fatal occupational injuries and illnesses was 122 per 10,000 full time workers requiring days off from work. There was a total of 4,002,700 recordable cases and 1,158, 900 cases which involved days off from work (BLS, 2009). Thus it is important to ensure the safety of workers.

Behavior Based Safety (BBS) blends in organization development, behavioral science and quality principles with safety management to reduce industrial illness and injuries (Krause, 2001). Sulzer-Azaroff and Austin (2000) identified the fundamental elements of BBS as follows: i. identify behaviors that are unsafe, ii. define these behavior in measurable terms, iii. develop and implement methods to measure those identified behaviors, iv. provide feedback; and v. reinforce progress.

Management plays a key role in BBS. Management systems must promote permanent change in favor of BBS to have lasting impact. Barrett (2000) identified three components of safety management triad: i. People, ii. environment; and iii. policies and procedures. The important mechanisms used by management to change behavior are observation, process design, goal type, and feedback.

Observation- Frequency and focus of observation affects the outcomes.  Frequency of observation refers to the rate of contact – daily, intermittent and weekly. Cooper (2009) revealed that daily contact reduced the number of injuries followed by intermittent contact. Weekly observation however, did not statistically have a significant impact. Focus of observation is either on workgroup, one-on-one, peer-to-peer or self observation approaches. Cooper (2009) identified workgroup observation has greater injury reduction followed by one-on-one observations. But workgroup observations are more effective in static industries whereas one-on-one is more effective in dynamic industries.

Process design structure- several mechanisms like goal setting, training, competition or incentives can be used.

Goal Setting- three kinds of goal setting can be used- participative, implicit and assigned. Cooper (2009) showed that implicit and participative goal setting has great impact in reducing injuries than the assigned goals. But out of the two behavior change had greater impact with participative rather than with implicit or assigned goals.

Feedback mechanism- four kinds of feedback mechanisms can be used: briefing, verbal, written and posted. Cooper (2009) identified that a combination of verbal, written and posted feedback given in a weekly briefing is more effective in reducing injuries and causing behavior changes.

Cameron and Duff (2007) identified three behavior change perspectives: cognitive (goal setting), behavioral (behavior modification) and eclectic (social learning).

Cognitive (goal setting). Figure 1 explains that changes in behavior can be caused by a desire to achieve the set goals by putting in efforts and having direction of these efforts.
 
Goal Setting


Figure 1. Goal setting process (Locke and Latham 1990)

 

Behavior modification. The dictionary defines behavior as function of its consequences. Figure 2 shows that the problem behavior must be identified, observed and recorded. This includes identifying the frequency of problem behavior, the use of appropriate interventions (use variables like praise/punish, tokens, goals/feedback or training), and then recording the frequency of the resulting behavior.

Organizational Behavior


Figure 2. Model of organizational behavior modification (Luthan and Kreitner 1974)

 

Social Learning. Figure 3 explains the integration of behavior modification and goal setting explains the role of antecedent’s i.e. goal and consequences or feedback in determining behavior. Social learning theory acknowledges the interplay between cognitive, behavioral and environmental factors.

Social Learning


Figure 3. Social learning perspective (Luthan 1982)

 

Feedback plays a very important role in any process chosen by management to reduce injuries and illness. Reinforcement results in increasing the probability of behavior to recur. Using feedback is advantageous as it can be used to effectively engage workers and makes them committed to build a positive culture compared to tangible reinforcement such as prizes, rewards, tokens etc. (Krause, 2001).

Sulzer-Azaroff and de Santamaria (1980) in their study identified eighteen unsafe conditions, developed intervention and provided feedback notes. Results showed a decrease in frequency of hazards by a mean of 60%. Sulzer-Azaroff (1997) also proved that simple written feedback improves safety practices.

Hopkins et al. (1986) aimed at improving air quality and increasing self-protective behaviors through corrective feedback. Air quality improved from 160 to 90 parts and also seven key self protective behaviors increased at the end of an intervention lasting nine days.

Komaki et al. (1982) studied the role of training and feedback to increase the safety in poultry plant with 200 employees who were divided among three departments. Safe performance increased from 78% to 75%, 83% and 90%. After intervention providing feedback the performance increased to 89%, 95% and 96%. Despite the improvements, it was also noted that attributing improvements solely to feedback would be misleading.

Cooper (2009) revealed in his study that a process that uses a combination of feedback mechanisms doubles the impact in reducing injuries and illness as opposed to using one or two mechanisms of feedback. This is because different people are responsive to different types of mechanisms. It is important to make the employees feel that they are part of the safety improvement process.

Definitions

Dynamic industry – Industries that are characterized with high rates of entries and exits (Burke and Hansley 2009).

Static industry- Industries that do not have high rates of entries and exits (Burke and Hansley 2009).

 Objectives

Safety measures through BBS are effective ways to reduce injuries and illness. Thus any process chosen to improve safety will use one or any combinations of feedback mechanisms to reinforce safe behavior. This systematic review aimed at studying the relationship between feedback mechanism and reducing the accidents, injuries and illness in both dynamic and static industries. It also aims at identifying the combination of feedback mechanisms that are used in static and dynamic industries.

Methods

Search strategy

Two channels were used in this study to identify articles during literature search: electronic and bibliographical sources. The online search was conducted using three different databases: ProQuest, EBSCOhost and Academic search. These databases were searched using the following keywords: behavioral based safety, BBS, feedback, static, intervention, management, safety, injury, accident, dynamic, written feedback and verbal feedback. Bibliographical sections in all related original articles were checked for studies relevant to the actual study.

Inclusion/exclusion criteria
Only articles written in English were included in this study. Articles must have used interventions and also must have used at least one form of feedback mechanism. Subjects used must have been from static or dynamic industries and articles must be epidemiologic studies.

No specific time period was chosen in order to access all available published papers. Articles were excluded if they are in other languages due to difficulties in translating other languages, or if they were descriptive and non-epidemiological studies.

Critical appraisal
Epidemiological Appraisal Instrument (EAI) (Genaidy et al. 2007) was used to assess the quality of all the accepted papers. The goal of the EAI is to “evaluate the design of the studies on basis of a critical appraisal system rooted in sound epidemiological principles”. EAI evaluates the study under five main headings: study description, subject selection, observation quality, data analysis and generalization of results and had a total of 43 questions. Each question was assigned an EAI scores between zero and two. Zero points were given if the information is not available, one point if partially available and two points if the information is available. EAI was conducted twice, each of them one week apart.

Results

Identification of studies
The electronic and bibliographical search conducted in September 2009 resulted in 22 articles related to feedback mechanism and a reduction in accident, injury and illness. With the inclusion criteria only eight articles were eligible for this review. Figure 4 explains the literature search strategy as well as the results.

Search Chart


Figure 4. Flow chart showing literature survey and search strategy.

 

Description of evidence
Table 1 summarizes a detailed description of all the articles included in this study. All the studies used intervention. Four articles were from static industries and the other four were from dynamic industries.

Study description- All the articles have stated the hypothesis clearly. The exposure variables and source of the population were also clearly described in all the articles, however, the outcomes were not clearly discussed in Winn, et al. (1999) and Cooper et al. (1994). No article mentions the adverse effects as a consequence of the intervention, covariates and confounder variables. Main findings are clearly described in all the articles.

Study’s methodological quality – Participant rate was unable to be determined in the following articles Cooper (2006), Winn et al. (1999), Cooper et al. (1994) and Komaki et al. (1980). Subjects were not randomized in any study. The method in which the subjects were chosen was not clearly mentioned in the articles.

Observational quality- Subjects were not blinded to the exposure in all the articles. It was not possible to determine the reliability of outcome measures. Methods of assessing the outcome variables were standard across all groups in every study that was conducted.

Data analysis- Winn et al. (1999) and Nasanen and Saari(1987) have not taken prior history of accidents, injuries or incidents events into account for analysis. Neither of the studies have made adequate adjustments for covariates and confounders in terms of individual variables or environment variables in the analysis.

Generalization- Results of the eight studies are not completely generalized and applied to the entire population.


Table 1. Table of evidence
Table of Evidence

 

Discussion

The critical appraisal rating for the articles are provided in figure 5. The overall and average scores of EAI in all five sub-sections were evaluated for all eight articles selected for this study after two rounds of blind appraisal. The intra-rater reliability score was evaluated by calculating the Kappa coefficients to show the agreement between the two critical appraisal processes. The simple and weighted Kappa coefficients were 0.6146 and 0.7214 respectively.


EAI Scores


Figure 5. EAI scores for included papers

 

Cooper (2006) focused on exploring the relationships between management commitments, observation frequency and safe score in a paper mill (static industry). Nine main departments with 500 unionized personnel were studied. The study design used in Cooper (2006) was: baseline 1(14 weeks) followed by intervention 2 (21 weeks) returning to baseline for 14 weeks continued with replication of intervention for 18 weeks and again return to baseline 3 for 4 weeks finally replicating the intervention for 19 weeks. Combinations of four feedback mechanisms were used in the study: verbal, written, graphical and briefings. Study revealed a 45% reduction in injuries. Cooper (2006) had scored the highest in data analysis with a score of 2 as it uses sound statistical analysis like range, standard deviation and correlations. Beside the injury reduction this study also described cost-benefit analysis and took the history of past accidents into consideration. However it scored only 0.8 as the articles did not clearly mention how the subjects were chosen or whether they were randomized.

Winn, et al. (1999) hypothesized that the number of vehicle crashes and USPS reportable incidents would reduce after the implementation of BBS program. Participating subjects in this study were 61 members of National Association of Letter Carriers (NALC) employed at the Morgantown branch. The study did not clearly specify the study design thus scored 0.6. The study used a combination of three feedback mechanisms: verbal, written and briefing. Previous accidents and incidents history were used to assess the effectiveness of the intervention. This study also did not clearly explain the data analysis. Study results showed that there was 79% decrease of total accidents.

Cooper, et al. (1994) aimed at proving that an inverse relationship exist between safety performance and accident rates. It also aimed to prove that a relationship exists between concurrent non-safety organizational variables and accident rates apart from studying the effects of goal-setting and performance feedback technique to improve safety in cellophane manufacturing industry (static).

Study was conducted in a single large manufacturing cellophane film site with 540 employees. The design of study was: i. management briefings, ii. recruiting observers, iii. training of observers, iv. establishing departmental baselines and v. establishing departmental goals. Safety performance and goal commitment was measured by using range and standard deviation. All the employees were considered as subjects in this study, and it scored 0.5 in study’s methodological quality. The study used combination of four feedback mechanisms: briefings, written, verbal and graphic feedback. It was found that there was 21% decrease in accident rate.

The study that Komaki, et al. (1980) conducted aimed to find whether training alone is sufficient to improve and maintain performance on the job, or whether it is necessary to provide feedback to maintain performance. There were 55 participants in this study from the vehicle maintenance division (static industry). A multiple baseline design with reversal component was used in this study with five conditions: i. baseline, ii. training only, iii. training and feedback, iv. second training and v. training and feedback.  This study proved that training alone is not necessary to improve safety performance. Providing feedback improves performance significantly. All employees from sweeper repair, light and heavy equipment repair, and preventive maintenance were chosen as participants and their characteristics were described. The study used a combination of two mechanisms: visual and briefings. The results of the study indicated lost time accidents declined to 0.4 accidents per month.
In static industries all the articles scored low in study’s methodological quality as the participants were chosen without being randomized and they were not blinded to the interventions. There was no adverse impact to participants due to intervention as it only increased safety. But all the studies clearly described their aim, hypothesis, outcome variables and exposure variables. Only Komaki, et al. (1980) used reversal component of checking whether the performance improved if feedback was not provided. However, Winn. et al. also tried to study the effects of feedback, whereas, Cooper, (2004) and Cooper et al. (1994) did not concentrate to see the effects of feedback on improving safety behavior but was used only as a variable along with intervention. On the whole none of the studies’ results could be generalized as subjects are from one organization and one industry thus making the sample exclusive. It is also clear that the static industry uses a maximum combination of four feedback mechanisms as no two people are same and different feedback mechanisms works for different people.

Mattitla, and Hyodynmaa(1988), aimed at proving that behavioral safety program improves safety even in difficult setting like construction (dynamic industry). This study used a control design. Multiple baselines were utilized to evaluate how targets and feedback influenced performance. Study was conducted in two settings – apartments and office building sites. Data was collected for over 20 weeks for the apartment site and for 22 weeks for the office building site. The effects of intervention were evaluated using information from two control sites. Combination of two feedback mechanisms were used – first written feedback was given which was later changed to visual feedback. In this article, two hundred and 220 participants were studied. The statistical analysis was not clearly given thus the article scored only 0.8. Study revealed that in office buildings when no feedback was given there was only 63% safety index, whereas with written feedback there was 73% and with visual feedback there was 78%. In the apartment setting when no feed feedback was given, there was only 74% safety index whereas with written feedback there was 84% and with visual feedback there was 88%. But the article does not explain why apartment and office building sites had different results when both were construction sites.

The purpose of Nasanen and Saari (1987) study was to evaluate the effects of behavior modification to reduce accidents. The study was conducted in a Finnish shipyard (dynamic industry) that had a total of 2000 employees but only 32 employees were selected for this study. This study scored 1.5 as all of the workers belong to trade unions that had shared interests thus making it comparable. But the study did not reveal the selection process. This study also scored 1.3 for observation quality as the process of observations: data recording and calculation of housekeeping index were clearly mentioned. Study design was baseline observation (7 weeks) – training seminar (1 hr) - feedback to foreman only (1 week period) – public graphic feedback (8 weeks) – follow up (7 weeks after end of feedback period). A combination of two feedbacks was used: visual and verbal feedback. This study revealed that the frequency rate of accidents fell by 17%. This study also scored 1.2 in data analysis as it clearly mentioned that it used t-test, past histories and its calculations.

Haynes, et al. (1982) study aimed at evaluating the effectiveness of intervention package which includes feedback, competition and incentives to reduce the accident rate of urban transit operators. It had a total of 100 urban transit operators (dynamic industry). This study scored 2 in study’s methodological quality as it clearly described how the participants were selected and had used randomization in selecting the participants. This study utilized ‘within group reversal design’ using multiple baselines including control group. Thus groups were comparable. This study also scored 2 in data analysis as it used past historical data of accidents and incidents, also used t-test. The result of this study showed 24.9% reduction in accidents. A combination of two feedback mechanisms were used in this study –briefing and visual.

Zhu, et al. (2000) study focused on showing the effects of behavior modification techniques in improving safety in the offshore oil drilling industry (dynamic industry). Participants of this study included 190 employees from three mobile offshore drilling units at Gulf of Mexico. It scored 1.4 in study description as hypothesis, outcomes, exposure variables were clearly mentioned along with sample. It also scored 1.3 in data analysis as the study clearly stated how the LTA incidence rate and total incidence rate were calculated. Historical data were also utilized. Study design included four phases: development of observation measures, development of baseline data through observation, training crew members and introducing goals and feedback. The results of study showed 90% employees working safely, but only one feedback mechanism was used – visual feedback.

Thus in dynamic industry overall all the articles have scored high on study description and methodological and data analysis. These articles clearly mentioned their aim, hypothesis, outcomes expected, exposure variables, samples, and data analysis methods. But only two studies utilized control design – Mattila and Hyodynmaa (1988) and Haynes. et al.(1982). Former proved that feedback does improve safety, whereas the latter showed that fading incentives except feedback and low incentives will improve safety. Reversal component was used in two articles only – Haynes, et al. (1982) and Nasanen and Saari (1987). In the dynamic industry articles it is clear that the maximum combination of two feedback mechanisms were used. Both in static industries and in dynamic industry the results were not generalized as the samples or subjects were from same organization in a particular industry.

Conclusion

This study revealed that feedback does play an important role in behavior modification and also for recurrence of safe behavior increasing the safety and reducing accidents, incidents and also claims to be paid by static or dynamic industries. Eight articles in this study have used various combinations of feedback mechanisms.

Only two articles used control design to isolate the effects of feedback on improving safety. It would have been more convincing and sounder if every article used controlled and experimental groups. Only three articles used reversal component which is also a good indicator to isolate the effect of feedback on improving safety.

Two articles in dynamic industry articles and three articles in static industry have focused on identifying a relationship between feedback and safety behaviors. The other articles have used feedback as a variable following an intervention.

It also clear that static industry has used combination of four feedback mechanisms and dynamic has used only a combination of two. In any combination, visual feedback is used in all eight articles. Thus future studies must aim to identify the best combination of feedback mechanisms to improve safety. It is also important for management to conduct a pilot study to identify the feedback mechanisms that will work for their employees.

Based on this systematic review when an intervention was conducted it was followed by feedback which expected two things, firstly, outcomes e.g. general safety, material handling, housekeeping, use of personal protective equipment and others. And secondly, the end results e.g. reduced fatalities, accidents, injuries and lost time accidents reduction. None of the articles took employee motivation and job satisfaction into consideration, which is necessary to increase safety performance. Finally, the results of the articles were not generalized as the subjects were from one organization from one industry in one location. Thus the results can be partially applied to the whole industry.

It is important for organization, managers and supervisors to understand a safety problem can be solved easily and at a reduced cost by giving feedback at appropriate times to the employees in a way that will best reach them. It is crucial to keep in mind that feedback alone cannot solve the safety problems. Feedback combined with interventions or goal settings or any other technique would result in great percentage of success.


References
Barrett, G A. 2000. Management's impact on behavior safety. Professional Safety. 26-28.

Bureau of Labor 2009. Injuries, Illnesses, and Fatalities. United States Department of Labor. http://www.bls.gov/iif/ [Retrieved Nov. 25, 2009]

Burke, A., & Hansley, A. 2009. Market concentration and business survival in static v dynamic industries. Kiel Working Papers, Kiel Institute for World Economy, 1517:1-21.

Cameron, I., and R. Duff. 2007. A critical review of safety initiatives using goal setting and feedback, Construction Management and Economics 25: 495-508.

Cooper, M. 2008. Behavioral Safety Interventions. Professional Safety 54 (2): 36-45.

Cooper, M. D. 2006. Exploratory analyses of the effects of managerial support and feedback consequences on behavioral safety maintenance. Journal of Organizational Behavior Management, 26(3): 1-41.

Cooper, M. D., Phillips, R. A., Sutherland, V. J., & Makin, P. J. 1994. Reducing accidents using goal setting and feedback: A field study. Journal of Occupational and Organizational Psychology, 67: 219-240.

Genaidy, A. M., LeMasters, G. K., Lockey, J., Succop, P., Deddens, J., Sobeih, T. and Dunning, K. 2007. Anepidemiological appraisal instrument - a tool for evaluation of epidemiological studies. Ergonomics, 50: 6, 920 – 960.

Haynes, R. S., Pine, R. C., & Fitch, G. H. 1982. Reducing accident rates with organizational behavior modification. Academy of Management Journal, 25(2): 407-416.

Hopkins, B. L., Conard, R., Dangel., R,Fitch, G., Smith,M.J., & Anger, W.K. 1986. Behavioral technology for reducing occupational exposures to styrene. Journal of Applied Behavioral Analysis, 19: 3-11.

Komaki, J., Heinzmann, A. T., & Lawson, L. 1980. Effect of training and feedback: Component analysis of a behvioral safety program. Journal of Applied Psychology, 65(3): 261-270.

Krause, T. R. 2001. Moving to the 2nd generation in behavior-based safety. Professional Safety, 27-32.

Locke, E.A. & Latham, G.P. 1990. A Theory of goal setting and task performance. Prentice Hall International (UK) Ltd, London.

Luthan, F. & Kreitner, R. 1974. Organizational behavior modification. Scott-Forsman, Glenview, IL.
Luthan, F. 1982. Improving performance: A behavioral problem solving approach, in Fredriksen, L.W. (ed.) Handbook of organizational management, John Wiley, New York.

Mattila, M. & Hyodynmaa, M. 1988. Promoting job safety in building: An experiment on the   behavior analysis approach. Journal of Occupational Accidents, 9: 255-267.

Nasanen, M. & Saari, J. 1987. The effects of positive feedback on housekeeping and accidents at a shipyard. Journal of Occupational Accidents, 8: 237-250.

Sulzer-Azaroff, B. 1997. Ten ways to heighten the safety culture of your organization. The Safety & Health Practitioner, 15 (7): 18-20.

Sulzer-Azaroff, B. & Austin, J. 2000. Does BBS work? Behavior-based safety and injury reduction: A survey of the evidence. Professional Safety, 19-24.

Sulzer-Azaroff, B., & de Santamaria, M. C. 1980. Industrial safety hazard reduction through performance feedback. Journal of Applied Behavior Analysis, 3: 287-295.

Winn, G. L., Frederick, L. J., & Church, G. M. 1999. Union and behavior-based safety: Always the odd couple? Professional Safety, 44(12): 32-34.

Zhu, Z., Wallin, J. A., & Reber, R. A. 2000. Safety improvements: An application of behavior modification techniques. Journal of Applied Management Studies, 9(1): 135-140.