Lately, the machine learning research witnessed a notable surge in interest towards State-Space Models (SSMs) as a viable alternative to the Transformer paradigm. The intrinsic strength of SSMs resides in their training and deployment capabilities: they can be trained utilizing methodologies akin to Convolutional Neural Networks (CNNs) or through parallel scan techniques, yet they can be deployed in a manner similar to Recurrent Neural Networks (RNNs), achieving inference in constant time. This is an ideal scenario since we mimic the operational characteristics of RNNs during the inference phase while maintaining the training efficiency/speed comparable to that of CNNs. Additionally, SSMs are capable of learning continuous-time parameters, which offer the flexibility to be discretized at any chosen time-step. While these models have demonstrated encouraging outcomes on simpler sequence modeling datasets and in a limited scope on standard computer vision tasks, their application within the realm of Reinforcement Learning (RL) remains unexplored. This project aims to investigate the potential of harnessing the capabilities of SSMs in the field of RL, aiming to construct efficient RL systems of the future.
Lately, the machine learning research witnessed a notable surge in interest towards State-Space Models (SSMs) as a viable alternative to the Transformer paradigm. The intrinsic strength of SSMs resides in their training and deployment capabilities: they can be trained utilizing methodologies akin to Convolutional Neural Networks (CNNs) or through parallel scan techniques, yet they can be deployed in a manner similar to Recurrent Neural Networks (RNNs), achieving inference in constant time. This is an ideal scenario since we mimic the operational characteristics of RNNs during the inference phase while maintaining the training efficiency/speed comparable to that of CNNs. Additionally, SSMs are capable of learning continuous-time parameters, which offer the flexibility to be discretized at any chosen time-step. While these models have demonstrated encouraging outcomes on simpler sequence modeling datasets and in a limited scope on standard computer vision tasks, their application within the realm of Reinforcement Learning (RL) remains unexplored. This project aims to investigate the potential of harnessing the capabilities of SSMs in the field of RL, aiming to construct efficient RL systems of the future.
Develop SSM-based RL framework that works in simulation, and conduct a detailed evaluation and analysis of such framework. Ideally, deploy this model in a real-world environment.
Develop SSM-based RL framework that works in simulation, and conduct a detailed evaluation and analysis of such framework. Ideally, deploy this model in a real-world environment.
Nikola Zubic (zubic@ifi.uzh.ch), Angel Romero Aguilar (roagui@ifi.uzh.ch)
Nikola Zubic (zubic@ifi.uzh.ch), Angel Romero Aguilar (roagui@ifi.uzh.ch)