Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Neural network architecture for legged locomotion
The project aims to investigate different neural network architectures used in control problems and design an architecture for neural network controller for ANYmal using deep reinforcement learning.
Keywords: Robotics, Deep reinforcement learning, Deep learning
The Robotic System Labs (RSL) is working on controlling the quadruped robot ANYmal using a neural network trained by model-free Deep Reinforcement Learning (DRL). As shown in [1], a locomotion policy emerging from model-free reinforcement learning method may exhibit asymmetric, unrealistic motions.
As many successful works in deep learning research employ application-specific architectures [2], [3], [4], we seek to develop an architecture for the locomotion task which results in faster learning and a more performant controller that can be deployed on the real robot. The participant will investigate neural network architectures used in control problems [4], [5], [6] and apply model-free deep reinforcement learning methods to evaluate the performance and learning rate of different architectures.
[1] Heess, Nicolas, et al. "Emergence of locomotion behaviours in rich environments." arXiv preprint arXiv:1707.02286 (2017). https://arxiv.org/abs/1707.02286
[2] Van Den Oord, Aäron, et al. "WaveNet: A generative model for raw audio." SSW. 2016.
[3] Gers, Felix A., Jürgen Schmidhuber, and Fred Cummins. "Learning to forget: Continual prediction with LSTM." (1999): 850-855.
[4] Zhang, He, et al. "Mode-adaptive neural networks for quadruped motion control." ACM Transactions on Graphics (TOG) 37.4 (2018): 145. https://dl.acm.org/citation.cfm?id=3201366
[5] Devin, Coline, et al. "Learning modular neural network policies for multi-task and multi-robot transfer." Robotics and Automation (ICRA), 2017 IEEE International Conference on. IEEE, 2017. https://arxiv.org/abs/1609.07088
[6] Zhang, Fangyi, et al. "Vision-based reaching using modular deep networks: from simulation to the real world." arXiv preprint arXiv:1610.06781 (2016).
The Robotic System Labs (RSL) is working on controlling the quadruped robot ANYmal using a neural network trained by model-free Deep Reinforcement Learning (DRL). As shown in [1], a locomotion policy emerging from model-free reinforcement learning method may exhibit asymmetric, unrealistic motions.
As many successful works in deep learning research employ application-specific architectures [2], [3], [4], we seek to develop an architecture for the locomotion task which results in faster learning and a more performant controller that can be deployed on the real robot. The participant will investigate neural network architectures used in control problems [4], [5], [6] and apply model-free deep reinforcement learning methods to evaluate the performance and learning rate of different architectures.
[1] Heess, Nicolas, et al. "Emergence of locomotion behaviours in rich environments." arXiv preprint arXiv:1707.02286 (2017). https://arxiv.org/abs/1707.02286
[2] Van Den Oord, Aäron, et al. "WaveNet: A generative model for raw audio." SSW. 2016.
[3] Gers, Felix A., Jürgen Schmidhuber, and Fred Cummins. "Learning to forget: Continual prediction with LSTM." (1999): 850-855.
[4] Zhang, He, et al. "Mode-adaptive neural networks for quadruped motion control." ACM Transactions on Graphics (TOG) 37.4 (2018): 145. https://dl.acm.org/citation.cfm?id=3201366
[5] Devin, Coline, et al. "Learning modular neural network policies for multi-task and multi-robot transfer." Robotics and Automation (ICRA), 2017 IEEE International Conference on. IEEE, 2017. https://arxiv.org/abs/1609.07088
[6] Zhang, Fangyi, et al. "Vision-based reaching using modular deep networks: from simulation to the real world." arXiv preprint arXiv:1610.06781 (2016).
- Review of relevant literature and concretely outline the problem.
- Design and implement several neural network architectures.
- Application of model-free RL algorithms
- Review of relevant literature and concretely outline the problem. - Design and implement several neural network architectures. - Application of model-free RL algorithms
- Hands-on experience in deep learning
- Excellent working knowledge of C++
- Experience with TensorFlow is a plus
- Experience in using RNN, CNN is a big plus
- Creativity
- Hands-on experience in deep learning - Excellent working knowledge of C++ - Experience with TensorFlow is a plus - Experience in using RNN, CNN is a big plus - Creativity