Feature 2

Developing Higher-level Cognitive Skills in a Fire Protection Course

Roger C. Jensen, Ph.D., CSP

Biographies

Roger C. Jensen, Ph.D., CSP
Professor
Safety, Health, and Industrial Hygiene Department
Montana Tech

 

 

Full Text

Introduction

In an ideal world, all instructors of safety and health courses would be masters of course subject matter as well as the theories and practices for effective teaching. In practice, however, most instructors are much stronger in one or the other. This paper provides an example of how some fundamental knowledge from educational experts can be useful for improving a traditional safety course.

Is there a problem with the way traditional safety and health (S&H) courses are taught? It is asserted by this author that S&H education, in general, places too much emphasis on acquisition and comprehension of facts at the expense of helping students develop higher-level cognitive abilities. This paper explains the basis for the assertion and reports an experience upgrading a traditional fire protection course to include more assignments involving the higher-level ability known in the education community as synthesis.

Cognitive Abilities

A foundation for understanding levels of mental abilities comes from an often-referenced handbook by Benjamin Bloom and colleagues (Bloom et al. 1956). Professor Bloom and his colleagues at the University of Chicago developed taxonomies for learning. Their approach began with classification of three domains: cognitive, affective, and psychomotor. Of these, the cognitive domain is emphasized in college instruction for careers in occupational S&H. Within the cognitive domain, Bloom and colleagues proposed the six levels of cognitive development listed in Table 1. These categories have been cited extensively in the educational literature to support planning curricula, courses, units, assignments, and tests. Although some revisions have been proposed, respect for the Bloom taxonomy has endured for over 50 years (Krathwohl 2002, 212-218; Betts 2008, 100).


Bloom Level

Brief Explanation

Safety Example

1. Knowledge

Ability to learn and recall information.

Learn rules, regulations, and terminology.

2. Comprehension

Understand topic well enough to explain, interpret, or extrapolate it.

Develop and deliver training for confined space entry.

3. Application

Take a general principle or method and apply it to a real-world situation.

Use a hierarchy of hazard controls to address a recognized hazard.

4. Analytical

Compare alternatives. Analyze issues.

Sort out pros and cons of two drug-testing policies. Perform a risk assessment of a system.

5. Synthesis

Create something new based on knowledge and comprehension of subject matter.

Create a fault tree or event tree. Write a safety policy.

6. Evaluation

Use criteria to judge or appraise something.

Use written criteria to review bids from companies seeking a sub-contract.

Table 1. Overview of Bloom levels with examples.

This paper uses the original Bloom taxonomy for four reasons. First, the proposed revisions may be subjects of debate among educational theorists for the foreseeable future. Second, the Bloom taxonomy is widely recognized throughout the educational community. Third, the Bloom taxonomy is sufficient for planning S&H courses. And fourth, the Bloom categories list synthesis. This particular cognitive skill is the primary one used to upgrade the fire protection course.

The Bloom categories are seen as progressive levels of learning. That is not to say each level is a discrete step which must be completed before starting the next. The levels involve overlapping and interacting mental activities (Krathwohl 2002, 212-213). For example, acquiring factual knowledge about a subject, and developing deeper comprehension of the subject, often involves a back-and-forth process between the first two Bloom levels.

This concept of progressive cognitive development forms the foundation for education from the elementary grades through high school. As students progress through the grades, educators are expected to help students move upward in the Bloom levels. When students first enter college, their abilities in the lower Bloom levels (1-3) far exceed their abilities in the higher levels (4-6). Thus, undergraduate students typically feel more comfortable with assignments requiring memorization and other lower level skills. During the college experience, students should have multiple opportunities to continue strengthening their lower-level abilities and grow abilities for the higher-levels.

The responsibility for pushing students upward in the Bloom levels lies with the faculty members who plan curricula and courses. A key requirement for success involves matching assignments to student abilities. In lower-division courses, assignments involving Bloom’s lower levels provide students with opportunities to succeed. In those two years, too many assignments involving Bloom’s higher levels can discourage these students and frustrate their instructor. In upper-division courses, students should be ready for assignments aimed at developing their higher-level cognitive abilities. The instructor who fills upper division courses with material involving the lower cognitive levels does a disservice to students. The junior and senior classes should include learning experiences designed specifically to help students develop their cognitive potential. The course modification described in this paper may be viewed as a case study of an attempt to strengthen the content of a junior-level course by adding some high-level challenges.

Applying Bloom Levels to S&H Courses

A useful tool for planning a course for targeted Bloom levels is known as a knowledge survey. Some geology professors have used a knowledge survey to strengthen their undergraduate courses with some higher-level challenges, e.g. an introductory course in geological sciences at Macalester College (Wirth and Perkins 2003, 12). Knowledge surveys consist of long lists of outcomes the instructor wants the students to take away from the course. Each item in the list is classified by Bloom level, and used to compute the proportion of the course involving each level. Knowledge surveys are phrased as outcomes, and the students are asked to rate their level of confidence in their ability to meet that outcome. By administering the same knowledge survey at the beginning and the end of the course, the instructor obtains data for computing gain in confidence for each outcome. Analyses of class average survey results are used to identify outcomes that show poor gains.

The process of preparing a list of course outcomes appears well suited for planning courses in which the instructor wishes to deliberately incorporate assignments involving the higher-level cognitive abilities. But having a suitable tool for course design does not provide a reason for S&H educators to deliberately incorporate assignments involving the higher-level cognitive abilities. Why might S&H educators wish to deliberately introduce more high-level challenges into their courses?

Clearly, college-level courses on the same subject can vary considerably among different instructors. Thus, an attempt to generalize about teaching style and effectiveness would be unfruitful. But, this author asserts that there are three characteristics of S&H education which contribute to an over-emphasis on factual knowledge and comprehension. First, when it comes to textbooks, it is clear that traditional textbooks and handbooks in occupational safety emphasize subject-matter content. There appears to be a very large portion of end-of-chapter exercises that require only the three lower-level abilities; specifically questions designed to assure the students read the chapter, comprehended the major points, and have some ability to apply the concepts, equations, and principles. Most textbooks provide little to push students to develop higher-level cognitive skills.

A second factor contributing to over-emphasis on factual knowledge involves the need to construct tests. It is easier to make and grade tests aimed at factual information. Test items like multiple-choice, true/false, and matching are suitable for assessing student knowledge of facts. But these test item formats are poorly suited for testing higher-level cognitive skills.

The third reason concerns the abilities S&H graduates need to have successful careers. When students are able to pass their required undergraduate course using the three lower level abilities, are they really prepared to perform in a professional position? It is suggested that all that textbook knowledge will be useful, but not sufficient to succeed. They will need solid skills for analysis, synthesis, and evaluation. Educators can contribute to the growth of these skills by deliberately incorporating suitable assignment into the upper-division courses.

Methods

The Original Course

Curricula in occupational safety and health typically include a course in fire protection. A fire protection course taught by the author for several years uses a book by Craig Schroll (Schroll 2002). It contains solid technical content, follows a logical organization, and presents the material in a reasonably interesting way. Most of the course follows the book and emphasizes learning the material presented. The course organization presents material in the following order.

  1. Examples of industrial fires
  2. Fire behavior
  3. Fire loss control programs
  4. Life safety: exits, evacuation plans, drills
  5. Reducing risk from fire starting and spreading
  6. Reducing damage by installed protection systems
  7. Reducing damage by use of portable extinguishers
  8. Planning for emergency response and business continuity
  9. Managing emergency teams and fire brigades
  10. Post-fire actions: investigations, insurance, business, media, human issues

Students are encouraged to learn the material through typical approaches, specifically : reading the textbook, answering homework questions from the textbook, attending lectures, using instructor’s lecture notes to stay organized, studying for three unit exams, studying for the final exam, taking the exams, going over their exams after grading, and participating on a student team to complete a project.

After assessing the course in terms of Bloom’s taxonomy, it became clear that the course made no significant contribution to developing the student’s higher-level cognitive abilities. In order to change this, the author decided to upgrade the course by providing experiences in higher cognitive levels.

Course Upgrade

The course upgrade involved multiple changes. The main goal was to add challenges involving synthesis. Specifically, the upgrade introduced the use of fault trees as a tool for students to synthesize the many facts and codes found in the textbook. Those students who normally rely on memorization to pass their first and second year courses were pushed to learn an entirely different way of understanding. They were challenged to synthesize the many fact statements in the textbook into a logical framework capturing the essence of the text material in a logical, graphical format. The hope was for students to develop confidence that they understand the logic behind the many fire protection codes, and from that understanding be less dependent on memorizing fire protection codes and safe practices. The revised course was deployed for the first time during the fall semester of 2008.

The skill of synthesis involves creating something based on a foundation of subject-matter knowledge and comprehension. The product of synthesis could be, for example, an essay, a new equation, a principle, or a model of a process. A core aspect of system safety analysis projects involves developing models of the system (Clemens 2009, 13-17). Examples are chemical process flow diagrams, fault trees, free-body diagrams, and graphic representations. The process of developing such models requires knowledge and comprehension of the system, followed by the creative process of synthesis. Fault tree models are particularly useful for analyzing causes of specific undesired events.

Fault trees are diagrams to explain how an undesired event can occur. The process of constructing a fault tree is a high-level cognitive activity. It requires initially acquiring a clear understanding of the system or process involved. From this, the analyst needs to start with a precisely defined top event, and methodically work downward to construct the tree. This requires a disciplined, spatial thought process quite different from abilities most students have already mastered, such as learning new information by reading text, memorizing, solving mathematics problems, answering exam questions, and writing papers. Thus, learning how to construct fault trees is seen as a potentially effective educational mechanism for helping students improve their abilities for synthesizing.

The upgraded course contained material on fault trees in general, and examples specific to fires. Students were challenged to develop their own fault trees as part of various homework assignments. An effort was made to begin with a very simple fault tree, and very gradually move into more complex ones. The initial fault tree represented the same thing as the well-known “fire triangle” image used for public education. Both simply depict the concept that starting a fire requires three elements: fuel, oxygen, and heat. Exhibit 1 shows the fire triangle with the corresponding fault tree. The top of the tree is a rectangle with the undesired fire event. Under the top event is an AND gate. The AND gate indicates that the fire will only occur if all three events beneath the AND gate occur. In the tree, the three shapes beneath the AND gate are shown as rectangles with a triangular transfer gate attached. The transfer gates identify branches the students will develop during the course.

 

Fire Triangle/Fault Tree
Exhibit 1. The fire triangle (left) and corresponding fault tree (right)

 

A slightly more advanced fault tree depicted a sustained fire. The fire rectangle was used to communicate that sustaining a fire requires four elements: supply of fuel, supply of an oxidizer, sufficient heat, and a chemical chain reaction. The fourth element, chain reaction, was shown as an oval to indicate it is a basic event with no further development necessary.

An example of an assignment to help students understand one of the many topics is provided here. When studying flammable material fires, students were assigned to extend the fuel branch of Exhibit 1 down to the level of explaining the lower and upper range of flammability (LFL and UFL, respectively). One approach is shown in Exhibit 2. At the top is a transfer gate to show where it connects to the tree in Exhibit 1. The top event in the branch matches the event box it supports. Under that event is an extra rectangle with no gate between them. This fault-tree technique is used for explaining an attribute of the rectangle above. In this case, the lower box indicates that the fuel source for this application must have the attribute of being a flammable vapor in the flammable range. Below the attribute rectangle is an AND gate to show that the flammable material will only be in the flammable range if two conditions are met. Students were assigned to develop such a tree. In the next class, some were asked to sketch theirs on the blackboard. This resulted in some lively class discussion. Interestingly, the best fault tree came from a student with a relatively low grade point average, while some of the normally top students proposed illogical trees. This experience, and others during the course, made the instructor realize that (1) synthesis skills are unnecessary for making high grades in first and second year courses; and (2) one must not assume that students have any sort of inherent ability to synthesize.

 

Fuel Element Tree
Exhibit 2. Fault tree branch for the fuel element of a flammable vapor fire.

 

As the class continued learning about flammable materials, students had assignments to see how fault trees can be used to explain the rationale for provisions in the codes and industrial practices described in the textbook. For example, the text and lecture notes indicate there are two distinct tactics for managing flammable liquids and vapors in a manner that prevents ignition. One is to maintain it so the vapor concentration is kept well below the LFL. The other is to keep the vapor concentration above the UFL. In past classes, many students simply memorized these two related statements. They can answer multiple-choice test questions using memorization, without appreciating the significance to industry. The fault tree assignment was intended to help students recognize the two tactics as logical extensions of their fault tree. Thus, the fault tree assignments in conjunction with the textbook and class discussions were exercises aimed at developing appreciation for fault trees and perhaps other logical system representations.

Measure of Student Confidence

A knowledge survey was developed to measure student confidence in their mastery of the course material. It consisted of a list of knowledge and abilities the instructor would like each student to have by the end of the course. For this course, the list had 261 items. Students rated each item with a number from the following rating scale.

The knowledge survey methodology has potential to enhance a college course in multiple ways. First, it provides the students with a roadmap of the entire course. They are introduced to all items on the survey the first day of class. Then the course covers the items in order. That makes the instructor’s expectations transparent to the students. Second, the instructor uses it to determine the proportion of the course involving each Bloom level. Third, it provides a guide for the instructor developing exams, and for students preparing to take them. Fourth, by administering it on both ends of the course, the gain in confidence can be determined. The gain in confidence serves as feedback for the instructor to identify strengths and weaknesses in the course. The knowledge survey is not suitable for measuring actual mastery of the topics by individuals or the whole class.

Developing the knowledge survey required considerable thought and time. It needed to include the entire course content; and each item required classification into one Bloom level. The distribution of items by Bloom level was 46, 125, 47, 15, 25, and 3, for the six levels, respectively. An example item for each Bloom level is provided in the following list.

When administering the survey, the instructor explained that responses would not count in their grade, their good faith effort to rate their actual level of confidence would be appreciated, and data would be used for making a pre-course to post-course comparison for the class as a whole. The analyses focused on the change in confidence from the pre to post course surveys. The following equations were used.

The mean ratings for the entire class of nine students were used. For each item, the Mean Gain in rating was the difference in the mean post-rating (R2) and the mean pre-rating (R1).

MeanGain = R2 – R1

For each item, the Maximum Gain for the class was calculated as the highest possible rating (3) minus the class mean pre-rating.

MaxGain = 3 – R1

The Normalized Gain compares the Mean Gain relative to the Max Gain.

NormalizedGain = MeanGain/MaxGain

For clarity, the Normalized Gain was multiplied by 100 to make it a percentage. Analyses were limited to descriptive values and graphs created with Minitab software.

Results

There are numerous ways to look at results from the knowledge surveys. One is an area graph as shown in Exhibit 3. It graphs Normalized Gain for all items in the knowledge survey. This type of graph is used to look for trends as the course proceeded from knowledge survey item 1 to 261. A trend made apparent from the graph is the lower gains in the range of items 243 to 254. This decline involved Chapter 9 on emergency teams and fire brigades. The lower gains suggest a need to improve the instruction on this chapter.

Gain vs. Survey Items
Exhibit 3. An area graph of Normalized Gain (%) versus the survey items in order.

A second useful graphic is a boxplot of Normalized Gain for each Bloom level. The boxplot in Exhibit 4 shows the inter-quartile ranges for each Bloom level. It appears that Bloom levels 1, 2, and 3 had similar results, with medians in the 85-89 percent range. In contrast, for the three higher cognitive levels, gains for Bloom level 5 items (synthesis) had highest median (80) percent. The Bloom levels 4 (analysis) and 6 (evaluation) had a lower medians of 67 and 65, respectively. The extra broad spread for Bloom level 6 was due to only having three items in that category.

Gain/Bloom Level
Exhibit 4.  Inter-quartile range of normalized gain for items within each Bloom level.

Discussion

This course-upgrade project was initiated to strengthen an undergraduate safety course by adding assignments involving higher-level thinking. This paper describes the educational theory and the technical methodology for the upgrade. The course upgrade was intended to help students gain more from the course than facts; specifically in the following areas.

  1. Improved abilities for synthesizing.
  2. A deeper understanding of established fire protection practices, by helping students see the logic behind the recognized practices for fire prevention and control.
  3. An ability to construct fault trees for a wide range of undesired occurrences.
  4. Recognition that heavy reliance on memorization is the hard way to learn occupational safety and health. An easier way is to think logically about the hazards and the various tactics for reducing risk. Once the logic is understood, the codes and standards become much easier to learn.

A major challenge for this course-upgrade project was to assess learning outcomes in order to identify aspects needing improvement. The preferred methodology for evaluating benefits of a course requires measures of student abilities before and after completing the course (Jensen 2005, 26-32). For this project, the instrument for the pre and post measurements was the knowledge survey. It provided an indication of gain in confidence for responding to the items.

The first year for this upgraded course is best viewed as a pilot study in use of the knowledge survey and the fault-tree assignments. Some changes needed for future offerings of the course were identified. One lesson learned was to not ask students to read and rate 261 items during a single class session. That left them with about 15 seconds per item. It is unlikely that they could give each item the attention needed in that time. Alternatives are to reduce the number of items to about half, or spread the ratings over two class periods. The later solution would take two class periods at the beginning and another two at the end, for a total of four class periods. Because class time is a precious commodity, the preferred solution is to reduce the number of items to about half.

Two challenges emerged involving grading the fault-tree assignments. First, fault-trees are challenging to grade. A clear grading rubric is required to clarify the instructor’s expectations and scoring criteria. Second, some normally good students were unsuccessful at learning how to make fault trees. Thus, it seems to this instructor that the ability to create fault trees should be a rather small percentage of the overall weighted grade.

ACKNOWLEDGEMENT: The author of this publication was partially supported by Training Grant Number T03-OH008630 from the Centers for Disease Control and Prevention/National Institute for Occupational Safety and Health. The contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIOSH.

Bibliography

Betts, Stephen C. 2008. Teaching and assessing basic concepts to advanced applications: Using Bloom's taxonomy to inform graduate course design. Academy of Educational Leadership Journal 12, (3): 99-106.

Bloom, B. S. (Ed )., M. D. Engelhart, E. J. Furst, W. H. Hill, and D. R. and Krathwohl. 1956. Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. Handbook I - Cognitive domain., eds. Benjamin S. Bloom, David R. Krathwohl. New York: David McKay.

Clemens, Pat. 2009. Modeling in system safety analysis -- A significant source of error. J. System Safety 45, (1): 13-7.

Jensen, Roger C. 2005. Safety training: Flowchart model facilitates development of effective courses. Professional Safety 50, (2): 26-32.
Krathwohl, David R. 2002. A revision of Bloom's taxonomy: An overview. Theory into Practice 41, (4): 212-218.

Schroll, R. C. 2002. Industrial fire protection handbook. 2nd ed. Boca Raton, FL: CRC.
Wirth, K. R., and Perkins, D. Knowledge surveys: An indispensible course design and assessment tool. in Macalester College [database online]. 2003. Available from http://www.macalester.edu/geology/wirth/WirthPerkins.pdf (accessed March 3, 2009).

-