Dorothea Müller: Social Meta-Learning

BCCN Berlin / Technische Universität Berlin

Abstract

Animals forage to survive. To bridge the gap between ethology and modern meta-learning tools, we propose the following framework: An animal (agent) forages and explores multiple environments in its lifetime, i.e. meta-learns on multiple multi-armed bandits per episode. Throughout an episode, it can observe the behaviour of its conspecifics (other agents called advisors). By learning to infer their traits quickly, the agent can thus utilize their demonstrations for efficient environment exploration. We call this type of learning ‘social meta-learning’ and show that it is beneficial when the environment structure is difficult. We operationalized the advisors’ traits as reliability and knowledgeability: How likely an advisor communicates according to its beliefs (reliability) and how likely its beliefs are correct (knowledgeability). By adapting a counterfactual reasoning measure, we show that only agents with reliable and knowledgeable advisors learn socially. Furthermore, the dependency on an advisor was highest at the beginning when a single advisor was present. Lastly, we demonstrate how employing attention mechanisms such as softmax and an adaptation of dot-product attention affect social information filtering and how this leads to a variety of behaviours. A combination of memory-based meta-learning with dot-product attention performed best.

 

Additional Information

Master Thesis Defense

 

Organized by

Prof. Henning Sprekeler   & Prof. Klaus Obermayer     / Lisa Velenosi