Performance on the Wechsler Abbreviated

Performance on the Wechsler Abbreviated Scale of Intelligence matrix-reasoning subtest, an index of fluid-reasoning ability, significantly mediated the relationship between age and model-based choice. A major component of fluid reasoning is the ability to identify associations between mental representations across distinct “dimensions”, often referred to as relational buy STF-62247 (Wright et al., 2008). As the number of relevant dimensions increases, the more complex the relation becomes (Christoff et al., 2001). For example, low-level integration may involve representing a simple characteristic of an object (e.g., shape A is a circle), while higher levels of integration may involve identifying relations between properties of multiple objects (e.g., shape B is also a circle, but a different color) or assessing the relationship between multiple relations (e.g., shape A is to B as shape C is to D). Like fluid-reasoning tasks, model-based choice requires the integration of learned relationships across multiple dimensions. In our sequential reinforcement-learning task, a model-based chooser must be able to prospectively integrate the transition probabilities between the first- and second-stage states with the reward probabilities associated with each second-stage stimuli. In fluid-reasoning puzzles that require considering two or more joint relations, children have been found to select answers at the same speed as adults, but with less accuracy (Crone et al., 2009; Vodegel Matzen et al., 1994). These findings suggest that children may not consider all the relevant dimensions of the problem before selecting an answer. Children in our reinforcement-learning task may similarly not recognize that recruiting transition knowledge at the first stage influences their later options, and therefore fail to integrate this knowledge into their evaluation at the first-stage. Working memory plays a significant role in both fluid reasoning (Kane and Engle, 2002) and model-based choice (Otto et al., 2013a, 2013b). A large body of research has shown that working memory improves with age (De Luca et al., 2003; Tamnes et al., 2013). In this study, nearly half of the participants, across all age groups, in whom we assessed working memory performed at ceiling. Thus, we failed to obtain a reliable index of working memory, precluding the ability to clearly characterize its contribution to developmental changes in reinforcement-learning strategy. Nonetheless, we observed a significant relationship between working memory and model-based choice, suggesting that age-related improvements in working memory may contribute to the development of model-based learning. Fluid reasoning, which mediated improvements in model-based choice in our sample, also depends on working memory ability. Thus, future studies, employing more robust assessments of working memory, will be necessary to dissociate the contributions of working memory and fluid reasoning to the recruitment of model-based learning. The neurocircuitry underlying the development of model-based learning has not been directly characterized. However, the neural substrates underlying the cognitive processes examined in this study have been previously explored. Statistical learning depends on medial temporal lobe structures, including the hippocampus (Davachi and DuBrow, 2015; Preston et al., 2004; Schapiro and Turk-Browne, 2015). Recent findings suggest that developmental improvements in statistical learning parallel hippocampal structural development (Schlichting et al., 2016). To inform goal-directed actions, these learned sequential representations must be integrated with learned associations between stimuli, actions, and past rewards, which depend in part on contributions from the striatum (Balleine and O’Doherty, 2010). Working memory and relational integration have been shown to depend on dorsolateral (Curtis and D’Esposito, 2003) and rostrolateral (Wright et al., 2008) prefrontal cortical regions respectively. Cortical maturation typically proceeds from posterior to anterior cortical regions, with the dorsolateral prefrontal cortex being one of the latest maturing regions (Gogtay et al., 2004; Shaw et al., 2008). This developmental trajectory mirrors the proposed organizational hierarchy of the prefrontal cortex, in which increasingly complex and abstract representations recruit more anterior regions (Badre and D\'Esposito, 2007; Dixon and Christoff, 2014). Thus, younger individuals, for whom rostral and dorsolateral prefrontal regions are not yet mature, may have greater difficulty integrating and recruiting the multi-dimensional cognitive representations on which model-based learning depends. More broadly, this literature suggests that elements of the integrated prefrontal-hippocampal-striatal circuits underpinning key component processes of model-based learning exhibit distinct, and often protracted, maturational trajectories. Direct examination of the age-related changes in these circuits will be necessary to further elucidate their specific functional roles in the developmental emergence of model-based choice.