Adrien Doerig + Christmas Party!

Title: LLMs as models of high level perceptual processing

 

Abstract: The brain extracts far more from vision than just objects: it encodes relations, context, and meaning. I will show that embeddings from large language models (LLMs) provide quantitative models of this complexity, aligning closely with brain activity evoked by natural scenes. These embeddings capture functional selectivity across cortex, enable reconstruction of captions from brain data, and provide a more brain-like representational format than many state-of-the-art alternatives. I will then look beyond our results to recent findings and ongoing work in my lab, and speculate that LLM embeddings may offer a general framework for modeling high-level perceptual processing in the human brain.