Claus Lang: Deep Learning of Optical Flow for the Prediction of Sensory Consequences of Actions in Artificial Agents

BCCN Berlin / TU Berlin

Human self-perception is not passive, but a product of many complex functions of the nervous system. The brain constantly makes predictions about the sensory effects of the motor actions it controls and compares them to the actual perceived effects. This predictive process is not innate, but develops in infants as they gain experience through interacting with the world. The model behind it, called forward model, is believed to play an important role in both the sense of agency, the subjective feeling of being aware and in control of one’s own actions, and the sense of object permanence, the understanding that objects continue to exist, even when they cannot be directly perceived at the moment. The idea behind this thesis is to enable the development of these basic cognitive skills in artificial agents. Towards that end, two deep convolutional neural networks are fed successively with data generated by the agent itself through exploration. They are thereby trained to predict the visual movement encoded as optical flow, that results from the motor actions of the agent. These predictions are then used to implement a sense of agency and a sense of object permanence, the effectiveness of which is verified in real-world robot experiments.

Additional Information

Master thesis defense in the International Master Program Computational Neuroscience.

Organized by

Verena Hafner/Robert Martin

Location

BCCN Berlin, lecture hall, Philippstr. 13 Haus 6, 10115 Berlin