February 17, 2023 at 15:00 in REC GS.08
A key function of brains is the abstraction and maintenance of information from the environment for later use. To this end, we are able to recognize an event or a sequence of events and learn to respond properly. The challenge is to learn to recognize both what is important to either remember or respond to, and also when to act. Reinforcement Learning (RL) is typically used to solve complex tasks: to learn the what. Over the last years, we have developed a biologically plausible neural network model that explains how neurons learn to represent task-relevant information in delayed response tasks where information from the environment has to be maintained for later use. We propose a plausible mechanism for learning this persistent activity as a form of working memory. Closely related is the when: to be responsive, an agent has to sample the environment often enough, which, unfortunately, makes learning much harder in an RL setting. We developed a continuous-time version of the working-memory neural network model by matching the actual representation learning in the what setting to a plausible model of action selection. This decouples actions, and their duration, from the internal time-steps in the neural RL model using a plausible action selection system. Together, the resultant model successfully learns to react to the significant events of continuous-time tasks that require working memory, without any pre-imposed specifications about the duration of the events or the delays between them.