Up-to-date information about the lab’s publications is also available via Google Scholar.
Do, Q., Kane, G.A., McGuire, J.T., & Scott, B.B. (2022). Assessing evidence accumulation and rule learning in humans with an online game. bioRxiv preprint. doi:10.1101/2022.02.19.481071
Evidence accumulation, how the brain integrates sensory information over time, is an essential component of perception and decision making. In humans, evidence accumulation is commonly modeled as a diffusion process in which noise accumulates linearly with the incoming evidence. However, recent studies in rodents have shown that during perceptual decision making, noise scales non-linearly with the strength of accumulated evidence. The question of whether nonlinear noise scaling also holds for humans has been clouded by differences in the methodologies typically used to collect and analyze human and rodent data. For example, whereas humans are typically given explicit instructions in these tasks, rodents are trained using feedback. Therefore, to evaluate how perceptual noise scales with accumulated evidence, we developed an online evidence accumulation game and nonverbal training pipeline for humans inspired by pulse-based evidence accumulation tasks for rodents. Using this game, we collected and analyzed behavioral data from hundreds of participants trained either with an explicit description of the relevant decision rule or merely with experiential feedback. Across all participants, performance was well described by an accumulation process, in which stimuli were integrated equally across time. Participants trained using feedback alone learned the game rules rapidly and used similar strategies to those who received explicit instructions. Decisions in both groups were influenced in similar ways by biases and perceptual noise, suggesting that explicit instructions did not reduce bias or noise in pulse-based accumulation tasks. Finally, by leveraging data across all participants, we show that perceptual noise during evidence accumulation was best described by a non-linear model of noise scaling, consistent with previous animal studies, but inconsistent with diffusion models widely used in human studies. These results challenge the conventional description of humans’ accumulation process and suggest that online games inspired by evidence accumulation tasks provide a valuable large-scale behavioral assessment platform to examine perceptual decision making and learning in humans. In addition, the feedback-based training pipeline developed for this game may be useful for evaluating perceptual decision making in human populations with difficulty following verbal instructions.
Lempert, K.M., Breslow, D., Peterson, T.D., Kable, J.W., & McGuire, J.T. (2022). Statistical information about reward timing is insufficient for promoting optimal persistence decisions. PsyArXiv preprint. doi:10.31234/osf.io/3yezf
When deciding how long to keep waiting for delayed rewards that will arrive at an uncertain time, different distributions of reward times dictate different optimal strategies for maximizing reward. For example, when reward timing distributions are heavy-tailed (e.g., waiting on hold with customer service) there is a point in time at which waiting is no longer advantageous because the opportunity cost of waiting is too high. Alternatively, when reward timing distributions have a more predictable time scale (uniform or Gaussian), it can be advantageous to wait as long as necessary for the reward to arrive. Although people learn to approximate optimal strategies, little is known about how this learning occurs. One possibility is that people form a general cognitive representation of the probability distribution that governs reward timing and then infer a strategy from that model of the environment. Another possibility is that they learn an action policy more directly, without explicitly representing the reward timing distribution. Here, in a series of studies in which participants decided how long to persist for delayed rewards before quitting, we provided participants with general information about the reward timing distribution in several ways. Whether the information was provided through counterfactual feedback (Study 1), previous exposure (Study 2), or description (Study 3), it did not obviate the need for direct, feedback-driven learning in the decision context. These results suggest that learning when to quit waiting for delayed rewards might depend on task-specific experience, not solely on general probabilistic reasoning.
Bakst, L. & McGuire, J.T. (in press). Experience-driven recalibration of learning from surprising events. Cognition.
Different contexts favor different patterns of adaptive learning. A surprising event that in one context would accelerate belief updating might, in another context, be downweighted as a meaningless outlier. Here, across two experiments, we examined whether participants performing a perceptual judgment task under spatial uncertainty (n=29, n=63) would spontaneously adapt their patterns of predictive gaze according to the informativeness or uninformativeness of surprising events in their current environment. Uninstructed predictive eye movements exhibited a form of metalearning in which surprise came to modulate event-by-event learning rates in opposite directions across contexts. Participants later appropriately readjusted their patterns of adaptive learning when the statistics of the environment underwent an unsignaled reversal. Although significant adjustments occurred in both directions, performance was consistently superior in contexts in which surprising events reflected meaningful change, potentially reflecting a bias toward interpreting surprise as informative. Overall, our results demonstrate remarkable flexibility in contextually adaptive metalearning.
Pan, J., Klimova, M., McGuire, J.T., & Ling, S. (2022). Arousal-based pupil modulation is dictated by luminance. Scientific Reports, 12, 1390. doi:10.1038/s41598-022-05280-1
Pupillometry has become a standard measure for assessing arousal state. However, environmental factors such as luminance, a primary dictator of pupillary responses, often vary across studies. To what degree does luminance interact with arousal-driven pupillary changes? Here, we parametrically assessed luminance-driven pupillary responses across a wide-range of luminances, while concurrently manipulating cognitive arousal using auditory math problems of varying difficulty. At the group-level, our results revealed that the modulatory effect of cognitive arousal on pupil size interacts multiplicatively with luminance, with the largest effects occurring at low and mid-luminances. However, at the level of individuals, there were qualitatively distinct individual differences in the modulatory effect of cognitive arousal on luminance-driven pupillary responses. Our findings suggest that pupillometry as a measure for assessing arousal requires more careful consideration: there are ranges of luminance levels that are more ideal in observing pupillary differences between arousal conditions than others.
Toro-Serey, C., Kane, G.A., & McGuire, J.T. (2021). Choices favoring cognitive effort in a foraging environment decrease when multiple forms of effort and delay are interleaved. Cognitive, Affective, & Behavioral Neuroscience. doi:10.3758/s13415-021-00972-z
Cognitive and physical effort are typically regarded as costly, but demands for effort also seemingly boost the appeal of prospects under certain conditions. One contextual factor that might influence choices for or against effort is the mix of different types of demand a decision maker encounters in a given environment. In two foraging experiments, participants encountered prospective rewards that required equally long intervals of cognitive effort, physical effort, or unfilled delay. Monetary offers varied per trial, and the two experiments differed in whether the type of effort or delay cost was the same on every trial, or varied across trials. When each participant faced only one type of cost, cognitive effort persistently produced the highest acceptance rate compared to trials with an equivalent period of either physical effort or unfilled delay. We theorized that if cognitive effort were intrinsically rewarding, we would observe the same pattern of preferences when participants foraged for varying cost types in addition to rewards. Contrary to this prediction, in the second experiment, an initially higher acceptance rate for cognitive effort trials disappeared over time amid an overall decline in acceptance rates as participants gained experience with all three conditions. Our results indicate that cognitive demands may reduce the discounting effect of delays, but not because decision makers assign intrinsic value to cognitive effort. Rather, the results suggest that a cognitive effort requirement might influence contextual factors such as subjective delay duration estimates, which can be recalibrated if multiple forms of demand are interleaved.
Morin, T.M., Chang, A.E., Ma, W., McGuire, J.T., & Stern, C.E. (2021). Dynamic network analysis demonstrates the formation of stable functional networks during rule learning. Cerebral Cortex, 252–293. doi:10.1093/cercor/bhab175
Variations in the functional connectivity of large-scale cortical brain networks may explain individual differences in learning ability. We used a dynamic network analysis of fMRI data to identify changes in functional brain networks that are associated with context-dependent rule learning. During fMRI scanning, naïve subjects performed a cognitive task designed to test their ability to learn context-dependent rules. Notably, subjects were given minimal instructions about the task prior to scanning. We identified several key network characteristics associated with fast and accurate rule learning. First, consistent with the formation of stable functional networks, a dynamic community detection analysis revealed regionally specific reductions in flexible switching between different functional communities in successful learners. Second, successful rule learners showed decreased centrality of ventral attention regions and increased assortative mixing of cognitive control regions as the rules were learned. Finally, successful subjects showed greater decoupling of default and attention communities throughout the entire task, whereas ventral attention and cognitive control regions became more connected during learning. Overall, the results support a framework by which a stable ventral attention community and more flexible cognitive control community support sustained attention and the formation of rule representations in successful learners.
Bakst, L., & McGuire, J.T. (2021). Eye movements reflect adaptive predictions and predictive precision. Journal of Experimental Psychology: General, 150, 915–929. doi:10.1037/xge0000977
Successful decision making depends on the ability to form predictions about uncertain future events. Existing evidence suggests predictive representations are not limited to point estimates but also include information about the associated level of predictive uncertainty. Estimates of predictive uncertainty have an important role in governing the rate at which beliefs are updated in response to new observations. It is not yet known, however, whether the same form of uncertainty-modulated learning occurs naturally and spontaneously when there is no task requirement to express predictions explicitly. Here, we used a gaze-based predictive inference paradigm to show that (a) predictive inference manifested in spontaneous gaze dynamics, (b) feedback-driven updating of spontaneous gaze-based predictions reflected adaptation to environmental statistics, and (c) anticipatory gaze variability tracked predictive uncertainty in an event-by-event manner. Our results demonstrate that sophisticated predictive inference can occur spontaneously and that oculomotor behavior can provide a multidimensional readout of internal predictive beliefs.
Botvinik-Nezer, R., Holzmeister, F., Camerer, C., Dreber, A., Huber, J., Johannesson, M., … & Schonberg, T. (2020). Variability in the analysis of a single neuroimaging dataset by many teams. Nature, 582, 84–88. doi:10.1038/s41586-020-2314-9
Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.
Babcock S., Howard M.W., & McGuire J.T. (2020). Time-conjunctive representations of future events. Memory & Cognition, 38, 672–682. doi:10.3758/s13421-019-00999-1
It is widely accepted that people can predict the relative imminence of future events. However, it is unknown whether the timing of future events is represented using only a “strength-like” estimate or if future events are represented conjunctively with their position on a mental timeline. We examined how people judge temporal relationships among anticipated future events using the novel Judgment of Anticipated Co-Occurence (JACO) task. Participants were initially trained on a stream of letters sampled from a probabilistically repeating sequence. During test trials, the stream was interrupted with pairs of probe letters and the participants’ task was to choose the probe letter they expected to appear in the stream during a lagged target window 4-6 items (4.3-8.5 seconds) in the future. Participants performed above chance as they gained experience with the task. Because the correct item was sometimes the more imminent probe letter and other times the less imminent probe letter, these results rule out the possibility that participants relied solely on thresholding a strength-like estimate of temporal imminence. Rather, these results suggest that participants held 1) temporally organized predictions of the future letters in the stream, 2) a temporal estimate of the lagged target window, and 3) some means to compare the two and evaluate their temporal alignment. Response time increased with the lag to the more imminent probe letter, suggesting that participants accessed the future sequentially in a manner that mirrors scanning processes previously proposed to operate on memory representations in the short-term judgment of recency task.
Kao, C.-H., Khambhati, A.N., Bassett, D.S., Nassar, M.R., McGuire, J.T., Gold, J.I., & Kable, J.W. (2020). Functional brain network reconfiguration during learning in a dynamic environment. Nature Communications, 11, 1682. doi:10.1038/s41467-020-15442-2
When learning about dynamic and uncertain environments, people should update their beliefs most strongly when new evidence is most informative, such as when the environment undergoes a surprising change or existing beliefs are highly uncertain. Here we show that modulations of surprise and uncertainty are encoded in a particular, temporally dynamic pattern of whole-brain functional connectivity, and this encoding is enhanced in individuals that adapt their learning dynamics more appropriately in response to these factors. The key feature of this whole-brain pattern of functional connectivity is stronger connectivity, or functional integration, between the fronto-parietal and other functional systems. Our results provide new insights regarding the association between dynamic adjustments in learning and dynamic, large-scale changes in functional connectivity across the brain.
Toro-Serey, C., Tobyne, S.M., & McGuire, J.T. (2020). Spectral partitioning identifies individual heterogeneity in the functional network topography of ventral and anterior medial prefrontal cortex. NeuroImage, 205, 116305. doi:10.1016/j.neuroimage.2019.116305
Regions of human medial prefrontal cortex (mPFC) and posterior cingulate cortex (PCC) are part of the default network (DN), and additionally are implicated in diverse cognitive functions ranging from autobiographical memory to subjective valuation. Our ability to interpret the apparent co-localization of task-related effects with DN-regions is constrained by a limited understanding of the individual-level heterogeneity in mPFC/PCC functional organization. Here we used cortical surface-based meta-analysis to identify a parcel in human PCC that was more strongly associated with the DN than with valuation effects. We then used resting-state fMRI data and a data-driven network analysis algorithm, spectral partitioning, to partition mPFC and PCC into “DN” and “non-DN” subdivisions in individual participants (n = 100 from the Human Connectome Project). The spectral partitioning algorithm identified individual-level cortical subdivisions that varied markedly across individuals, especially in mPFC, and were reliable across test/retest datasets. Our results point toward new strategies for assessing whether distinct cognitive functions engage common or distinct mPFC subregions at the individual level.
Nassar, M.R., McGuire, J.T., Ritz, H., & Kable, J.W. (2019). Dissociable forms of uncertainty-driven representational change across the human brain. Journal of Neuroscience, 39, 1688–1698. doi:10.1523/JNEUROSCI.1713-18.2018
Environmental change can lead decision makers to shift rapidly among different behavioral regimes. These behavioral shifts can be accompanied by rapid changes in the firing pattern of neural networks. However, it is unknown what the populations of neurons that participate in such “network reset” phenomena are representing. Here we examined 1) whether and where rapid changes in multivariate activity patterns are observable with fMRI during periods of rapid behavioral change, and 2) what types of representations give rise to these phenomena. We did so by examining fluctuations in multi-voxel patterns of BOLD activity from male and female human subjects making sequential inferences about the state of a partially observable and discontinuously changing variable. We found that, within the context of this sequential inference task, the multivariate patterns of activity in a number of cortical regions contain representations that change more rapidly during periods of uncertainty following a change in behavioral context. In motor cortex, this phenomenon was indicative of discontinuous change in behavioral outputs, whereas in visual regions the same basic phenomenon was evoked by tracking of salient environmental changes. In most other cortical regions, including dorsolateral prefrontal and anterior cingulate cortex, the phenomenon was most consistent with directly encoding the degree of uncertainty. However, in a few other regions, including orbitofrontal cortex, the phenomenon was best explained by representations of a shifting context that evolve more rapidly during periods of rapid learning. These representations may provide a dynamic substrate for learning that facilitates rapid disengagement from learned responses during periods of change.
Lempert, K.M., McGuire, J.T., Hazeltine, D.B., Phelps, E.A., & Kable, J.W. (2018). The effects of acute stress on the calibration of persistence. Neurobiology of Stress, 8, 1–9. doi:10.1016/j.ynstr.2017.11.001
People frequently fail to wait for delayed rewards after choosing them. These preference reversals are sometimes thought to reflect self-control failure. Other times, however, continuing to wait for a delayed reward may be counterproductive (e.g., when reward timing uncertainty is high). Research has demonstrated that people can calibrate how long to wait for rewards in a given environment. Thus, the role of self-control might be to integrate information about the environment to flexibly adapt behavior, not merely to promote waiting. Here we tested effects of acute stress, which has been shown to tax control processes, on persistence, and the calibration of persistence, in young adult human participants. Half the participants (n = 60) performed a task in which persistence was optimal, and the other half (n = 60) performed a task in which it was optimal to quit waiting for reward soon after each trial began. Each participant completed the task either after cold pressor stress or no stress. Stress did not influence persistence or optimal calibration of persistence. Nevertheless, an exploratory analysis revealed an “inverted-U” relationship between cortisol increase and performance in the stress groups, suggesting that choosing the adaptive waiting policy may be facilitated with some stress and impaired with severe stress.
McGuire, J.T., & Kable, J.W. (2016). Deciding to curtail persistence. In K. D. Vohs & R. F. Baumeister (Eds.), Handbook of Self-Regulation: Research, Theory, and Applications (pp. 533–546). Guilford.
Krastev, S., McGuire, J.T., McNeney, D., Kable, J.W., Stolle, D., Gidengil, E., & Fellows, L.K. (2016). Do political and economic choices rely on common neural substrates? A systematic review of the emerging neuropolitics literature. Frontiers in Psychology, 7, 264. doi:10.3389/fpsyg.2016.00264
The methods of cognitive neuroscience are beginning to be applied to the study of political behavior. The neural substrates of value-based decision-making have been extensively examined in economic contexts; this might provide a powerful starting point for understanding political decision-making. Here, we asked to what extent the neuropolitics literature to date has used conceptual frameworks and experimental designs that make contact with the reward-related approaches that have dominated decision neuroscience. We then asked whether the studies of political behavior that can be considered in this light implicate the brain regions that have been associated with subjective value related to “economic” reward. We performed a systematic literature review to identify papers addressing the neural substrates of political behavior and extracted the fMRI studies reporting behavioral measures of subjective value as defined in decision neuroscience studies of reward. A minority of neuropolitics studies met these criteria and relatively few brain activation foci from these studies overlapped with regions where activity has been related to subjective value. These findings show modest influence of reward-focused decision neuroscience on neuropolitics research to date. Whether the neural substrates of subjective value identified in economic choice paradigms generalize to political choice thus remains an open question. We argue that systematically addressing the commonalities and differences in these two classes of value-based choice will be important in developing a more comprehensive model of the brain basis of human decision-making.
McGuire, J.T., & Kable, J.W. (2015). Medial prefrontal cortical activity reflects dynamic re-evaluation during voluntary persistence. Nature Neuroscience, 18, 760–766. doi:10.1038/nn.3994
Deciding how long to keep waiting for future rewards is a nontrivial problem, especially when the timing of rewards is uncertain. We report an experiment in which human decision makers waited for rewards in two environments, in which reward-timing statistics favored either a greater or lesser degree of behavioral persistence. We found that decision makers adaptively calibrated their level of persistence for each environment. Functional neuroimaging revealed signals that evolved differently during physically identical delays in the two environments, consistent with a dynamic and context-sensitive reappraisal of subjective value. This effect was observed in a region of ventromedial prefrontal cortex that is sensitive to subjective value in other contexts, demonstrating continuity between valuation mechanisms involved in discrete choice and in temporally extended decisions analogous to foraging. Our findings support a model in which voluntary persistence emerges from dynamic cost/benefit evaluation rather than from a control process that overrides valuation mechanisms.
McGuire, J.T.,* Nassar, M.R.,* Gold, J.I., & Kable, J.W. (2014). Functionally dissociable influences on learning rate in a dynamic environment. Neuron, 84, 870-881. doi:10.1016/j.neuron.2014.10.013
Maintaining accurate beliefs in a changing environment requires dynamically adapting the rate at which one learns from new experiences. Beliefs should be stable in the face of noisy data, but malleable in periods of change or uncertainty. Here we used computational modeling, psychophysics and fMRI to show that adaptive learning is not a unitary phenomenon in the brain. Rather, it can be decomposed into three computationally and neuroanatomically distinct factors that were evident in human subjects performing a spatial-prediction task: (1) surprise-driven belief updating, related to BOLD activity in visual cortex; (2) uncertainty-driven belief updating, related to anterior prefrontal and parietal activity; and (3) reward-driven belief updating, a context-inappropriate behavioral tendency related to activity in ventral striatum. These distinct factors converged in a core system governing adaptive learning. This system, which included dorsomedial frontal cortex, responded to all three factors and predicted belief updating both across trials and across individuals.
McGuire, J.T. & Kable, J.W. (2014). Go means green. Nature Neuroscience, 17, 489-490. doi:10.1038/nn.3680
News & Views commentary on this paper by Tom Schonberg, Akram Bakkour, and colleagues.
A simple cued-approach training procedure can bias economic choices toward specific goods. It appears to work by drawing overt attention toward trained items, scaling up their judged value.
Kool, W., McGuire, J.T., Wang, G.J., & Botvinick, M.M. (2013). Neural and behavioral evidence for an intrinsic cost of self-control. PLoS ONE, 8, e72626. doi:10.1371/journal.pone.0072626
The capacity for self-control is critical to adaptive functioning, yet our knowledge of the underlying processes and mechanisms is presently only inchoate. Theoretical work in economics has suggested a model of self-control centering on two key assumptions: (1) a division within the decision-maker between two ‘selves’ with differing preferences; (2) the idea that self-control is intrinsically costly. Neuroscience has recently generated findings supporting the ‘dual-self’ assumption. The idea of self-control costs, in contrast, has remained speculative. We report the first independent evidence for self-control costs. Through a neuroimaging meta-analysis, we establish an anatomical link between self-control and the registration of cognitive effort costs. This link predicts that individuals who strongly avoid cognitive demand should also display poor self-control. To test this, we conducted a behavioral experiment leveraging a measure of demand avoidance along with two measures of self-control. The results obtained provide clear support for the idea of self-control costs.
Bartra, O.,* McGuire, J.T.,* & Kable, J.W.* (2013). The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. NeuroImage, 76, 412–427. doi:10.1016/j.neuroimage.2013.02.063
Numerous experiments have recently sought to identify neural signals associated with the subjective value (SV) of choice alternatives. Theoretically, SV assessment is an intermediate computational step during decision making, in which alternatives are placed on a common scale to facilitate value-maximizing choice. Here we present a quantitative, coordinate-based meta-analysis of 206 published fMRI studies investigating neural correlates of SV. Our results identify two general patterns of SV-correlated brain responses. In one set of regions, both positive and negative effects of SV on BOLD are reported at above-chance rates across the literature. Areas exhibiting this pattern include anterior insula, dorsomedial prefrontal cortex, dorsal and posterior striatum, and thalamus. The mixture of positive and negative effects potentially reflects an underlying U-shaped function, indicative of signal related to arousal or salience. In a second set of areas, including ventromedial prefrontal cortex and anterior ventral striatum, positive effects predominate. Positive effects in the latter regions are seen both when a decision is confronted and when an outcome is delivered, as well as for both monetary and primary rewards. These regions appear to constitute a “valuation system,” carrying a domain-general SV signal and potentially contributing to value-based decision making.
McGuire, J.T., & Kable, J.W. (2013). Rational temporal predictions can underlie apparent failures to delay gratification. Psychological Review, 120, 395–410. doi:10.1037/a0031910
An important category of seemingly maladaptive decisions involves failure to postpone gratification. A person pursuing a desirable long-run outcome may abandon it in favor of a short-run alternative that has been available all along. Here we present a theoretical framework in which this seemingly irrational behavior emerges from stable preferences and veridical judgments. Our account recognizes that decision makers generally face uncertainty regarding the time at which future outcomes will materialize. When timing is uncertain, the value of persistence depends crucially on the nature of a decision maker’s prior temporal beliefs. Certain forms of temporal beliefs imply that a delay’s predicted remaining length increases as a function of time already waited. In this type of situation, the rational, utility-maximizing strategy is to persist for a limited amount of time and then give up. We show empirically that people’s explicit predictions of remaining delay lengths indeed increase as a function of elapsed time in several relevant domains, implying that temporal judgments offer a rational basis for limiting persistence. We then develop our framework into a simple working model and show how it accounts for individual differences in a laboratory task (the well-known “marshmallow test”). We conclude that delay-of-gratification failure, generally viewed as a manifestation of limited self-control capacity, can instead arise as an adaptive response to the perceived statistics of one’s environment.
McGuire, J.T., Cohen, J.D., & Botvinick, M.M. (2013). Mental effort. In H. Pashler (Ed.), Encyclopedia of the Mind (pp. 502–506). Thousand Oaks, CA: SAGE Publications. doi:10.4135/9781452257044.n186
McGuire, J.T., & Kable, J.W. (2012). Decision makers calibrate behavioral persistence on the basis of time-interval experience. Cognition, 124, 216–226. doi:10.1016/j.cognition.2012.03.008
A central question in intertemporal decision making is why people reverse their own past choices. Someone who initially prefers a long-run outcome might fail to maintain that preference for long enough to see the outcome realized. Such behavior is usually understood as reflecting preference instability or self-control failure. However, if a decision maker is unsure exactly how long an awaited outcome will be delayed, a reversal can constitute the rational, utility-maximizing course of action. In the present behavioral experiments, we placed participants in timing environments where persistence toward delayed rewards was either productive or counterproductive. Our results show that human decision makers are responsive to statistical timing cues, modulating their level of persistence according to the distribution of delay durations they encounter. We conclude that temporal expectations act as a powerful and adaptive influence on people’s tendency to sustain patient decisions.
Ribas-Fernandes, J.J.F., Solway, A., Diuk, C., McGuire, J.T., Barto, A.G., Niv, Y., & Botvinick, M.M. (2011). A neural signature of hierarchical reinforcement learning. Neuron, 71, 370–379. doi:10.1016/j.neuron.2011.05.042
Human behavior displays hierarchical structure: simple actions cohere into subtask sequences, which work together to accomplish overall task goals. Although the neural substrates of such hierarchy have been the target of increasing research, they remain poorly understood. We propose that the computations supporting hierarchical behavior may relate to those in hierarchical reinforcement learning (HRL), a machine-learning framework that extends reinforcement-learning mechanisms into hierarchical domains. To test this, we leveraged a distinctive prediction arising from HRL. In ordinary reinforcement learning, reward prediction errors are computed when there is an unanticipated change in the prospects for accomplishing overall task goals. HRL entails that prediction errors should also occur in relation to task subgoals. In three neuroimaging studies we observed neural responses consistent with such subgoal related reward prediction errors, within structures previously implicated in reinforcement learning. The results reported support the relevance of HRL to the neural processes underlying hierarchical behavior.
Kool, W.,* McGuire, J.T.,* Rosen, Z.B., & Botvinick, M.M. (2010). Decision making and the avoidance of cognitive demand. Journal of Experimental Psychology: General, 139, 665–682. doi:10.1037/a0020198
Behavioral and economic theories have long maintained that actions are chosen so as to minimize demands for exertion or work, a principle sometimes referred to as the law of less work. The data supporting this idea pertain almost entirely to demands for physical effort. However, the same minimization principle has often been assumed also to apply to cognitive demand. The authors set out to evaluate the validity of this assumption. In 6 behavioral experiments, participants chose freely between courses of action associated with different levels of demand for controlled information processing. Together, the results of these experiments revealed a bias in favor of the less demanding course of action. The bias was obtained across a range of choice settings and demand manipulations and was not wholly attributable to strategic avoidance of errors, minimization of time on task, or maximization of the rate of goal achievement. It is remarkable that the effect also did not depend on awareness of the demand manipulation. Consistent with a motivational account, avoidance of demand displayed sensitivity to task incentives and covaried with individual differences in the efficacy of executive control. The findings reported, together with convergent neuroscientific evidence, lend support to the idea that anticipated cognitive demand plays a significant role in behavioral decision making.
McGuire, J.T., & Botvinick, M.M. (2010). Prefrontal cortex, cognitive control, and the registration of decision costs. Proceedings of the National Academy of Sciences of the United States of America, 107, 7922–7926. doi:10.1073/pnas.0910662107
Human choice behavior takes account of internal decision costs: people show a tendency to avoid making decisions in ways that are computationally demanding and subjectively effortful. Here, we investigate neural processes underlying the registration of decision costs. We report two functional MRI experiments that implicate lateral prefrontal cortex (LPFC) in this function. In Experiment 1, LPFC activity correlated positively with a self-report measure of costs as this measure varied over blocks of simple decisions. In Experiment 2, LPFC activity also correlated with individual differences in effort-based choice, taking on higher levels in subjects with a strong tendency to avoid cognitively demanding decisions. These relationships persisted even when effects of reaction time and error were partialled out, linking LPFC activity to subjectively experienced costs and not merely to response accuracy or time on task. In contrast to LPFC, dorsomedial frontal cortex—an area widely implicated in performance monitoring—showed no relationship to decision costs independent of overt performance. Previous work has implicated LPFC in executive control. Our results thus imply that costs may be registered based on the degree to which control mechanisms are recruited during decision-making.
McGuire, J.T., & Botvinick, M.M. (2010). The impact of anticipated cognitive demand on attention and behavioral choice. In B. Bruya (Ed.), Effortless attention: A new perspective in the cognitive science of attention and action (pp. 103–120). Cambridge, MA: MIT Press.
Botvinick, M.M., Huffstetler, S., & McGuire, J.T. (2009). Effort discounting in human nucleus accumbens. Cognitive, Affective, & Behavioral Neuroscience, 9, 16–27. doi:10.3758/CABN.9.1.16
A great deal of behavioral and economic research suggests that the value attached to a reward stands in inverse relation to the amount of effort required to obtain it, a principle known as effort discounting. In the present article, we present the first direct evidence for a neural analogue of effort discounting. We used fMRI to measure neural responses to monetary rewards in the human nucleus accumbens (NAcc), a structure previously demonstrated to encode reference-dependent reward information. The magnitude of accumbens activation was found to vary with both reward outcome and the degree of mental effort demanded to obtain individual rewards. For a fixed level of reward, the NAcc was less strongly activated following a high-demand for effort than following a low demand. The magnitude of this effect was noted to correlate with preceding activation in the dorsal anterior cingulate cortex, a region that has been proposed to monitor information-processing demands and to mediate in the subjective experience of effort.
Mitchell, K.J., Raye, C.L., McGuire, J.T., Frankel, H., Greene, E.J., & Johnson, M.K. (2008). Neuroimaging evidence for agenda-dependent monitoring of different features during short-term source memory tests. Journal of Experimental Psychology. Learning, Memory, and Cognition, 34, 780–790. doi:10.1037/0278-73220.127.116.110
A short-term source monitoring procedure with functional magnetic resonance imaging assessed neural activity when participants made judgments about the format of 1 of 4 studied items (picture, word), the encoding task performed (cost, place), or whether an item was old or new. The results support findings from long-term memory studies showing that left anterior ventrolateral prefrontal cortex (PFC) is engaged when people make source attributions about reflectively generated information (cognitive operations, conceptual features). The findings also point to a role for right lateral PFC in attention to perceptual features and/or familiarity in making source decisions. Activity in posterior regions also differed depending on what was evaluated. These results provide neuroimaging evidence for theoretical approaches emphasizing that agendas influence which features are monitored during remembering (e.g., M. K. Johnson, S. Hashtroudi, & D. S. Lindsay, 1993). They also support the hypothesis that some of the activity in left lateral PFC and posterior regions associated with remembering specific information is not unique to long-term memory but rather is associated with agenda-driven source monitoring processes common to working memory and long-term memory.
Johnson, M.K., Mitchell, K.J., Raye, C.L., McGuire, J.T., & Sanislow, C.A. (2006). Mental rubbernecking to negative information depends on task context. Psychonomic Bulletin & Review, 13, 614–618. doi:10.3758/BF03193971
We previously demonstrated mental rubbernecking during the simple cognitive act of refreshing a just activated representation. Participants saw two neutral and one negative word presented simultaneously and, 425 msec later, were cued to mentally refresh (i.e., think of) one of the no-longer-present words. They were slower to refresh a neutral word than the negative word (Johnson et al., 2005, Experiment 6A). The present experiments extended that work by showing mental rubbernecking when negative items were sometimes the target of refreshing, but not when negative items were present but never the target of refreshing, indicating that expectations influence mental rubbernecking. How expectations might modulate the impact of emotional distraction is discussed.
* denotes equal contribution.