MODELLING TIME/ACCURACY TRADE-OFFS IN WHEN-TO-ACT DECISIONS
Decision making, time-accuracy trade-offs, bayesian belief updating, signal detection theory, microworld experiments
The pace and complexity of many of today's work domains - such as air traffic control, transportation, or manufacturing - requires operators to manage multiple tasks and to adapt to a dynamic environment. Analysis and design of such systems therefore requires a detailed understanding of the temporal properties of the physical system, the task, the environment and the agents. While some temporal problems such as multi-tasking and decision-making under time pressure have received much attention, other aspects are less well understood. These include the role of time perception in decision-making and control, the control of time-lagged systems, sequence errors (e.g. omission, commission, revision or repetition), duration errors such as temporal overshoot or undershoot, duration neglect in judgment and decision-making, interruption scheduling, human scheduling performance, and temporal awareness (see De Keyser, 1995, for a review; research summary Time design for a discussion of the functional role of time; Hildebrandt & Rantanen, 2004 for research problems in Time design).
This study is concerned with a particular aspect of time in decision-making, namely biases in the management of accuracy-timeliness trade-offs. For many diagnostic tasks, the quality of diagnostic decisions increases over time, while the probability of successfully executing the action decreases. For instance, the more pronounced a patient's symptoms become, the more certain we can be that the patient is suffering from a particular disease. At the same time, the longer the diagnostic decision is postponed, the more difficult it may become to treat the disease. Kerstholt (1994) used a similar cover story in an experimental study and found that in conditions of high time pressure (rapid deterioration of the controlled process), participants' use of judgment-oriented strategies (requesting additional information about the cause of a problem) led to an increase in system failures. This finding suggests that decision-makers may be biased towards improving accuracy at the expense of timeliness. In the current experiment, the incentive to postpone a decision was induced by an alarm that appeared at some point during the trial and would help to improve the quality of the decision. Based on Kerstholt's (1994) findings, over-reliance on the alarm in situations where a decision should be taken immediately was expected, so that decisions would frequently be late.
In this study (Hildebrandt & Meyer, 2005), we developed a formal model combining signal detection theory and bayesian belief updating to explore the strategies that would optimise the probability of success. This model provided the standard against which we compared behavioural data gathered in microworld experiments (other microworld simulations developed in DIRC include the PumpPlant scenario). In the experiment, participants could take decisions either on the basis of a-priori information, or wait for an alarm that would appear later in the trial. The alarm would improve their decision quality, but the later they took the decision, the less likely it was that the action would be executed successfully. Alarm reliability and timing were manipulated as independent variables. Contrary to previous studies, results suggest that participants often favoured decision timeliness over accuracy in situations where the overall decision success could have been improved by waiting for the alarm. The factors that contribute to this bias may include the variability of alarm timing, misperception of elapsed time, misjudgment of the change in action probability, and over-reliance on one of the information sources due to misjudgment of its predictive value.
Control decisions and alarm responses in dynamic real-time systems are not isolated, one-off events, but are embedded in a process of information seeking and receiving, belief updating, and action. A common problem in such situations is the when-to-act dilemma, where a decision can be taken on current information, or alternatively further information could be sampled. Unless the temporal properties of the task, the environment, the available information sources, and the temporal control heuristics of the operator, are analysed in greater detail, this important class of temporal error phenomena may be neglected in system analysis and design. Formal modelling of the decision plays an important role in this process by providing a standard against which performance can be compared. Microworld experimentation has proved a valuable tool for investigating complex human behaviour under controlled conditions. This line of research may provide useful insights into the role of alarms by focusing not only on an operator's reaction to an alarm, but on the prospect of receiving the alarm, and the reasoning involved in deciding whether to wait for it or not.
Microworld When-to-act [requires Firefox]
Hildebrandt, M., and Meyer, J. (2005). When to act? Managing time-accuracy trade-offs in a dynamic belief updating task. In Proceedings of the 49th Annual Meeting of the Human Factors and Ergonomics Society.
Hildebrandt, M. & Rantanen, E. (2004). Time Design. Proceedings
of the 48th Annual Meeting of the Human Factors and Ergonomics Society
De Keyser, V. (1995). Time in ergonomics research. Ergonomics, 38, 1639-1660.
Kerstholt, J.H. (1994). The effect of time pressure on decision-making behaviour in a dynamic task environment. Acta Psychologica, 86, 89-104.
Michael Hildebrandt [hilde at cs dot york dot ac dot uk]
|Page Maintainer: firstname.lastname@example.org||Credits||Project Members only||Last Modified: 10 August, 2005|