Anton Komissarov: Deep Neuroevolution as a Method to Produce a Population of Behaviorally Different Agents for Use in Multiagent Reinforcement Learning Scenarios

BCCN Berlin / Technische Universität Berlin

Abstract

 

The real world around us, and including ourselves, is a very complex and unpredictable environment. However, if we want to incorporate machines into our world, we must formalize it and partially reproduce it. Multiagent reinforcement learning is a framework that gives us tools to abstract something in the form of a model of an environment, means to observe and change the environment, and agents which learn from interactions with the environment. When trained together, agents become constrained to the roles they assumed, and if paired with previously unseen agents, may fail to adapt. Exposing an agent to a set of other behaviorally different agents minimizes the risks of the agent becoming too specialized. However, a collection of behaviorally different agents is expensive to produce with traditional methods that train only a single agent simultaneously. We show that deep neuroevolution can be used to create such a set of agents in a single training procedure. Our results demonstrate that exposure to a set of behaviorally different agents makes the behavior of an agent subjected to the re-training more robust and less specific. We anticipate that deep neuroevolution can be used in other MARL scenarios, for example, to expose an autonomous driving system to a variety of road users before it is allowed to enter the real world.

 

Additional Information

Master Thesis Defense

 

Organized by

Prof. Klaus Obermayer   & Prof. Manfred Opper     / Lisa Velenosi