Hence, by comparing neural activity between these trial types, the authors are able to isolate responses

caused by PPEs. How, then, would the brain respond to a pseudo-reward prediction Fludarabine supplier error? A number of possibilities seemed reasonable. Hierarchical organization is already thought to exist in the lateral prefrontal cortex, with more rostral regions representing more abstract and temporally extended plans (make ganache) and more caudal regions executing more concrete and immediate actions (snap chocolate bar) (Koechlin et al., 2003). Might hierarchical PPE mechanisms utilize this existing hierarchy? Alternatively, representations of specific goals and outcomes can be found in the ventromedial prefrontal and orbitofrontal (Burke et al., 2008) cortices. Might these same regions update subgoal representations? In a series of three experiments, the authors demonstrate activity that is instead consistent with a third hypothesis: neural responses to pseudo-reward prediction errors show remarkable similarity to familiar RPE responses. Using EEG, previous studies have shown RPE correlations in a characteristic midline voltage

wave termed the feedback-related negativity (FRN; Holroyd and Krigolson, 2007). In the current study, this same negative deflection can be seen in response to a PPE. The source of the FRN is often assumed to lie in the dorsal anterior cingulate cortex (ACC), and, when the hierarchical task is taken into the MRI scanner, PPE-related Adenylyl cyclase activity is indeed found in the ACC BOLD signal (Ribas-Fernandes et al., Afatinib in vivo 2011). While reward prediction errors

can be found in single-unit activity in the ACC (Matsumoto et al., 2007), the current observation by Ribas-Fernandes et al. (2011) that pseudo-rewards, as well as fictive rewards (Hayden et al., 2009), cause similar activity requires a theory of ACC processing that goes beyond simple reward-and-error processing. One suggestion is that activity in the region is more concerned with behavioral update caused by the outcome than caused by the reward prediction error per se (Rushworth and Behrens, 2008). Further similarities can be found in subcortical structures. PPEs, like RPEs, are coded positively in the ventral striatum and negatively in the habenular complex. Although it is not yet clear whether the reported PPE activity recruits the dopaminergic mechanisms famous for coding RPEs, this latter finding makes it a likely possibility. Cells in the monkey lateral habenula not only code RPEs negatively, but they also causally inhibit the firing of dopamine cells in the ventral tegmental area (Matsumoto and Hikosaka, 2007). The data presented in Ribas-Fernandes et al. (2011) therefore raise the possibility that prediction error responses at different levels of a hierarchical learning problem recruit the same neuronal mechanisms.