Probabilistic vs. non-probabilistic approaches to the neurobiology of perceptual decision-making
Highlights
► Optimal decision making requires accumulation of evidence to bound in confidence. ► Behavioral/neuronal evidence suggests humans and animals follow such strategy. ► Drift-diffusion and attractor network models approximate this strategy. ► Probabilistic population codes have potential to implement strategy precisely.
Introduction
Efficient decision-making requires inferring the state of the world from uncertain or ambiguous evidence [1]. Little evidence results in inaccurate decisions, such that it is in the decision maker's best interest to boost her confidence by accumulating evidence over time and, if possible, across cues before committing to a decision. Thus, it is essential for the decision maker to perform decisions in two stages: first, she accumulates evidence to reach a certain level of confidence, and – once this level is reached – commits to her decision (Figure 1). A hunting eagle, for example, needs to be fairly certain about the presence of a rabbit before initiating its dive. Similarly, humans require certainty about the state of the surrounding traffic before crossing the street. In both of these cases a period of evidence accumulation is followed by acting upon this evidence.
We first give a short overview over the origins of statistically optimal, two-stage decision-making – which we will refer to as the normative strategy – as well as behavioral evidence that humans follow such a strategy. This provides us with a set of properties that decision-making models need to feature, and with respect to which we compare three types of models: diffusion models, models based on attractor dynamics, and probabilistic population codes. We show that the model based on probabilistic population codes provide a neural implementation of the normative model of decision making, while the other approaches provides various approximation to the normative approach.
Section snippets
Decision making under uncertainty
The realization that decision-making is essentially a task of probabilistic inference based on uncertain information was pioneered by Pascal in his famous wager on the benefits and losses involved in believing in God [2]. Gaining popular following, Pascal's approach was extended to all sorts of probabilistic decision problems, such as Bernoulli's well-known St. Petersburg paradox [3] that deals with wagering in games of chance. Finally, Laplace fully formalized general decision making under
Optimal accumulation of evidence
Behavioral studies have confirmed that human observers do not only take uncertainty into account, but also do so close-to-optimally according to the two-stage procedure outlined above. In the stage of evidence accumulation across time and cues, the decision maker needs to weight the momentary evidence in proportion to its reliability. Cue integration experiments that modulate the reliability of one of the cues have confirmed that humans indeed take this reliability into account [1, 10, 11],
Committing to a decision at the bound
The second stage of optimal decision-making under uncertainty is to commit to a decision once a pre-set level of certainty, called the decision bound, has been reached. Setting this bound optimally is complex, as it depends on both factors internal to the decision maker and properties of the task at hand. Thus, we first consider evidence for the presence of such a bound, and then discuss its optimality.
In reaction time tasks, decision makers are able to trade-off their decision speed with the
Diffusion decision models for 2AFC decision-making
The dominant model of decision-making for two-alternative forced choice (2AFC) tasks in psychology is the diffusion decision model [DDM [24••, 26, 41, 42]], in which a particle drifts and diffuses between two boundaries (Figure 2). Hitting either of these boundaries triggers a decision. A decision is correct if the particle hits the boundary corresponding to the mean drift rate. Incorrect choices occur due to the stochastic particle diffusion, and are less frequent for large drift rates. Thus,
Decision-making by attractor dynamics
Many neural models of decision making are based on networks with attractor dynamics. As we will argue, these models only approximate diffusion decision models and, as such, might not be optimal even when decisions are binary and the evidence is of constant reliability. Nonetheless, these models have the advantage of incorporating many biological features, such as different types of neurotransmitter receptors and distinct classes of excitatory and inhibitory neurons [47]. Their dynamics is best
Decision-making with probabilistic population codes
To recap, probabilistic decision-making requires the decision maker to at least maintain a representation of the certainty with which either option is correct throughout both stages of the decision process. Fortunately, neural population codes seem to be well suited to not only represent this certainty, but also full probability distributions over the stimulus [45••, 54, 55, 56] (Figure 4). Furthermore, as long as the neural spike variability belongs to the exponential family with linear
Conclusions
Based on behavioral and neural data of humans and animals, we have argued for following a normative approach to the modeling of decision-making. To this respect, we have pointed out weaknesses of both diffusion models as well as decision models based on attractor networks, in particular with respect to the representation of reliability and confidence. Probabilistic population codes, on the contrary satisfy requirements posed by the normative approach, and can explain both behavioral and neural
References and recommended reading
Papers of particular interest, published within the period of review, have been highlighted as:
• of special interest
•• of outstanding interest
Acknowledgements
A.P. was supported by grants from the National Science Foundation (BCS0446730), a Multidisciplinary University Research Initiative (N00014-07-1-0937) and the James McDonnell Foundation.
References (61)
- et al.
Variance as a signature of neural computations during decision making
Neuron
(2011) - et al.
The cost of accumulating evidence in perceptual decision making
J Neurosci
(2012) - et al.
Reward rate optimization in two-alternative decision making: empirical tests of theoretical predictions
J Exp Psychol Hum Percept Perform
(2009) - et al.
A comparison of sequential sampling models for two-choice reaction time
Psychol Rev
(2004) Decision making in recurrent neuronal circuits
Neuron
(2008)- et al.
Synaptic computation underlying probabilistic inference
Nat Neurosci
(2010) - et al.
Neurobiological computational systems
Pattern recognition and the responses of sensory neurons
Ann N Y Acad Sci
(1969)- et al.
Reading population codes: a neural implementation of ideal observers
Nat Neurosci
(1999) - et al.(1996)
Pensées
Specimen theoriae novae de mensura sortis
Théorie analytique des probabilités
Paper on statistics of repetitions
The National Archives of the UK: HW 25/38
Studies in the History of Probability and Statistics. XXXVII. A. M. Turing's Statistical Work in World War II
Biometrika
Sequential analysis
Optimum character of the sequential probability ratio test
Ann Math Stat
The Statistical Research Group, 1942–1945
J Am Stat Assoc
Integration of proprioceptive and visual position-information: an experimentally supported model
J Neurophysiol
Humans integrate visual and haptic information in a statistically optimal fashion
Nature
Dynamic reweighting of visual and vestibular cues during self-motion perception
J Neurosci
Neural correlates of reliability-based cue weighting during multisensory integration
Nat Neurosci
The analysis of visual motion: a comparison of neuronal and psychophysical performance
J Neurosci
Neuronal correlates of a perceptual decision
Nature
Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment
J Neurosci
Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task
J Neurosci
Decision processes in visual perception
Neural correlates, computation and behavioural impact of decision confidence
Nature
Representation of confidence associated with a decision by neurons in the parietal cortex
Science
Implicit knowledge of visual uncertainty guides decisions with asymmetric outcomes
J Vis
Cited by (28)
The statistics of optimal decision making: Exploring the relationship between signal detection theory and sequential analysis
2021, Journal of Mathematical PsychologyCitation Excerpt :A recent development in perceptual decision-making research is the recognition that in the repeated trial version of the computational task, the optimal decision threshold that maximizes the reward rate is non-linear and time-varying rather than constant. In this case, dynamic programming techniques are used to derive the optimal decision boundaries, and the precise nature of the boundary time-dependency is related to experimental factors such as inter-trial delay times or time penalties for errors (Drugowitsch & Pouget, 2012; Malhotra et al., 2018; Tajima et al., 2019). It has been pointed out, however, that optimality in these cases depends on a linear costing of time; in the more ecologically valid case where rewards are time discounted, DDM-like models with collapsing boundaries are not optimal (Marshall, 2019; Steverson et al., 2019).
Confidence Representation of Perceptual Decision by EEG and Eye Data in a Random Dot Motion Task
2019, NeuroscienceCitation Excerpt :Furthermore, the drift diffusion model (DDM) has been widely and successfully applied to the performance and statistics of the RT in human decision making (Ratcliff and Smith, 2004; Bogacz et al., 2006; Van den Berg et al., 2016b). Accordingly, although some studies discussed the complexity of confidence and furthermore introduced innovative solutions to fit confidence data to the DDM (Moreno-Bote, 2010; Drugowitsch and Pouget, 2012; Yeung and Summerfield, 2012), however, there are also studies endeavored to find and demonstrate the classic DDM parameters in correlation with certainty level (Ratcliff and Starns, 2009, 2013; Philiastides et al., 2014). EEG data which record the electrical activity of the brain has also contributed to the study of decision confidence.
Confidence as Bayesian Probability: From Neural Origins to Behavior
2015, NeuronCitation Excerpt :Accumulation-to-bound models can account not only for choice and reactions times, but also for decision confidence (Fetsch et al., 2014; Kepecs et al., 2008; Vickers et al., 1985). The theory of probabilistic population codes can provide a normative algorithm for integration of evidence over time that may be optimal for action selection under a large range of conditions (Beck et al., 2008; Drugowitsch and Pouget, 2012). In these models, a summary confidence level (akin to the LRP) can be computed using linear integration of neural activity (Beck et al., 2008; Drugowitsch and Pouget, 2012).
Emerging principles of population coding: In search for the neural code
2014, Current Opinion in NeurobiologyCitation Excerpt :Here we will focus on the latter. Bayesian approaches and probabilistic population codes are not discussed here, for recent reviews see [9,10]. The utility of quantifying the accuracy of a specific readout algorithm is that it can be compared with the psychophysical accuracy of the animal; if the accuracy of the readout cannot account for the psychophysical accuracy, then the specific readout can be rejected.