Dynamics of recurrent neural networks with piecewise linear activation function in the context-dependent decision-making task

Purpose. This paper aims to elucidate the dynamic mechanism underlying context-dependent two-alternative decision-making task solved by recurrent neural networks through reinforcement learning. Additionally, it seeks to develop a methodology for analyzing such models based on dynamical systems theory. Methods. An ensemble of neural networks with piecewise linear activation functions was constructed. These models were optimized using the proximal policy optimization method. The trial structure, featuring constant stimuli over extended periods, allowed us to treat inputs as system parameters and consider the system as autonomous during finite time intervals. Results. The dynamic mechanism of two-alternative decision-making was uncovered and described in terms of attractors of autonomous systems. The possible types of attractors in the model were characterized, and their distribution within the ensemble of models relative to the cognitive task parameters was studied. A stable division into functional populations was observed in the ensemble of models, and the evolution of these populations’ composition was examined. Conclusion. The proposed approach enables a qualitative description of the problem-solving mechanism in terms of attractors, facilitating the study of functional model dynamics and identification of populations underlying dynamic objects. This methodology allows for tracking the evolution of system attractors and corresponding populations during the learning process. Furthermore, based on this understanding, a two-dimensional network was developed to solve a simplified context-free two-alternative decision problem.  

recurrent neural network, reinforcement learning, cognitive task, Attractor, Population dynamics