Michael Frank: Tuning striatal dopamine signals to optimize reinforcement learning across tasks
Brown University, USA
The basal ganglia and dopaminergic (DA) systems are well studied for their roles in reinforcement learning, with a rich ties to machine learning. However, a closer look at the biology suggests an architecture that differs from that typically assumed in artificial agents, whereby RPEs are scalar and globally influence downstream targets. Computational and empirical considerations suggest that DA signals may be enriched to support structured striatal learning across different tasks. First, neuromodulated Hebbian plasticity within opponent striatal D1 vs D2 pathways, when combined with a "meta-critic" that drives adaptive dopamine levels across tasks, can provide robust advantages over traditional monolithic RL models over a range of environments. These models suggest that empirical observations of altered learning and decision making in patient populations reflect a byproduct of an otherwise normative mechanism. Second, dopamine RPE signals are not global and synchronous across the striatum. Rather, spatiotemporal dynamics (in the form of traveling waves of dopaminergic activity) provide a mechanism to support credit assignment to appropriate striatal subregions. I will present experimental data and computational models suggesting that such dynamics enable agents to reinforce striatal subregions that are most well tuned to the generative task statistics, facilitating adaptive structured behavior.
Guests are welcome!
Sophie Fromm & Milena Musial / Lisa Velenosi
Location: Online via Zoom - please send an email to email@example.com for access