Abstract:
Introduction: This paper represents an attempt to evaluate the effectiveness of the Department of Defense's Human Factors Analysis and Classification (DOD-HFACS) system. DOD-HFACS delineates the human factors causes of aviation mishaps, and is based upon Wiegmann and Shappell's HFACS coding system. The main difference from HFACS is that DOD-HFACS includes an additional level of fine-grain classification. This layer consists of 147 detailed nanocodes that are to be used to identify the mishap causes. Method: The internal validity, external validity, and utilitarian criteria of DOD-HFACS were evaluated. A total of 123 naval aviators used DOD-HFACS to identify the human factors causes of two aviation mishap scenarios. Results: There were reasonable levels of inter-rater reliability in the majority of the nanocodes that were not considered to be causal to the mishaps. However, for those nanocodes of which at least half of the raters thought applied to the mishaps, acceptable levels of inter-rater reliability were not achieved. Similarly, when compared to `expert' reference ratings, there were acceptable levels of agreement for those nanocodes that were discarded, but not for the majority of the nanocodes that were identified by the experts as being causal to the mishap. Discussion: This study has identified that more parsimony, increased mutual exclusivity, and training are required to use DOD-HFACS effectively.
