Wulfram Gerstner, École Polytechnique Fédéral de Lausanne

Synaptic plasticity controlled by surprise

During Hebbian plasticity,  pre- and postsynaptic activity work together to cause a change of the weights. However, neuromodulators signaling reward, novelty, or suprise influence synaptic plasticity as well. We therefore have to consider in models of synaptic plasticity a total of three factors (pre, post, modulator) that control learning [1]. Such three-factor rules have also been called neo-Hebbian [2]. While the role of neuromodulators related to reward is well studied in theories of reinforcement learning, a theory of surprise-driven learning is missing. Here we discuss theories of surprise that can serve as starting point for a framework of surprise-based learning.

Two components are needed in a  framework of surprise-based learning [3]: (i) a confidence-adjusted surprise measure to capture environmental statistics as well as subjective beliefs, (ii) a surprise-minimization learning rule, or SMiLe-rule, which dynamically adjusts the balance between new and old information without making prior assumptions about the temporal statistics in the environment. We apply our framework to a dynamic decision making task and a maze exploration task to demonstrate that it is suitable for learning in complex environments, even if the environment undergoes gradual or sudden changes. A synaptic implementation of learning  in a network of spiking  neurons with hidden neurons provides additional insights [4].

[1] N. Fremaux and W. Gerstner (2016), Front. Neural Circuits 9:85 doi: 10.3389/fncir.2015.00085
[2] Lisman et al. (2011), Trends in Neurosciences 34: 536–547. doi: 10.1016/j.tins.2011.07.006
[3] M. Faraji, K. Preschoff, W. Gerstner, (2016) http://arxiv.org/abs/1606.05642
[4] D.J. Rezende and W. Gerstner (2014), Front. Comput. Neurosci. http://dx.doi.org/10.3389/fncom.2014.00038

Organized by

Henning Sprekeler