December 13, 2022 at 15:00
Deep learning models have become important tools in the cognitive neuroscience of language, vision, mathematics, music and many other domains. In particular, internal states of these deep learning models can successfully be used to predict (with “encoder-decoder models”) the brain activation that one can observe using brain imaging techniques. However, it is unclear whether such successful predictions also lead to a better understanding of how the brain processes language, images, math or music. Are the observed alignments a mere curiosity, or can they inform theories of cognitive processing? In this talk, I argue that the encoder-decoder framework must be accompanied by efforts to ‘open the blackbox’ if we want such work to contribute to theories of cognitive processing. I will present some of our own work on language processing, and discuss representational stability analysis, as well as the attribution, attention-tracking and diagnostic probing methods we have used to understand the inner working of deep learning models (including LSTMs and Transformers).