Dr. Laura Ikuma of Louisiana State University followed-up her previous study on ways to measure display effectiveness by creating a manual for member companies to apply. Measures utilized in the first study on students were replicated with plant operators. Eye-tracking hardware and software were purchased by the COP as part of this project.
This project had three major aims:
1. Validate the use of performance metrics developed in Phase 1 in actual operators
2. Determine the impact of trend graphs on situation awareness and performance using a virtual plant and actual operators
3. Determine operator preference in using trend graphs versus snapshot parameters values.
- Can the metrics developed in Phase 1 be used with actual operators?
- Can the metrics developed in Phase 1 be used to reliability detect differences between significantly different user interfaces when completing typical control-room operation scenarios?
- Can situation awareness be measured in actual operators using the SAGAT approach?
- Do trend graphs improve situation awareness and performance within the virtual plant?
- Do operators subjectively find the graphs useful, and would they be willing to use them in real operations? What aspects of the graphs do they find most useful?
The Phase 2 work began investigating the effects of performance shaping factors on operator performance using the metrics and experiment guidelines defined in Phase 1. These metrics can measure operator performance, workload levels, eye movements, and situation awareness. Whereas Phase 1 did not attempt to determine performance differences for novel applications, Phase 2 will add a new concept (trend graphs) to the virtual plant and measure the effect on performance-shaping factors using the metrics from Phase 1.
- Analysis and results of experimental testing of performance metrics using actual operators at the COP Fall 2012 meeting presentation
- Recommendations for performance measurement using the performance metrics developed in Phase 1 and 2 at COP Fall 2012 meeting presentation
- Revised experiment guide document for COP members to use in designing, executing, and analyzing their own interface and operator testing