Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Semantic Segmentation of Off-Road Scenes for Autonomous Robotic Construction
The goal of this project is to investigate a deep learning framework for off-road scene semantic segmentation with special application to construction sites.
Scene understanding is a key step in the development of navigation algorithms for autonomous vehicles, becoming especially challenging in off-road, unstructured environments such as construction sites. In these scenarios, semantic knowledge about the surrounding terrain and obstacles is of vital importance for planning optimum paths, and can also be used to inform vehicle driving parameters to ensure safe and efficient operation.
In recent years, research in the domain of scene understanding for autonomous driving has seen enormous progress due to the success of Convolutional Neural Networks (CNNs) in numerous image recognition tasks. However, there is still very little work applying deep learning techniques to the more challenging off-road environment. One of the main limitations here is the large amount of annotated data required by these models during the learning process to perform accurately. Though there are some publicly available benchmark suites containing millions of labeled images, they are usually restricted to generic objects or urban road scenes and do not contain off-road navigation-relevant classes such as terrain types (gravel, mud, sand, grass, obstacle, etc).
The aim of this project is to overcome this drawback through transfer learning, which involves taking an existing CNN architecture, that is pre-trained for urban road-scene understanding, and investigating a novel fine-tuning architecture for the task of classifying terrain types in off-road scenes, assessing the network performance during the training cycle.
This work is part of the NCCR Digital Fabrication, a large Swiss project involving multiple disciplines that aims to lead the development and integration of digital technologies within the field of architecture. A successful method will directly be used within the semantic mapping pipeline developed for this project and integrated with a robotic excavator for outdoor operation.
Scene understanding is a key step in the development of navigation algorithms for autonomous vehicles, becoming especially challenging in off-road, unstructured environments such as construction sites. In these scenarios, semantic knowledge about the surrounding terrain and obstacles is of vital importance for planning optimum paths, and can also be used to inform vehicle driving parameters to ensure safe and efficient operation.
In recent years, research in the domain of scene understanding for autonomous driving has seen enormous progress due to the success of Convolutional Neural Networks (CNNs) in numerous image recognition tasks. However, there is still very little work applying deep learning techniques to the more challenging off-road environment. One of the main limitations here is the large amount of annotated data required by these models during the learning process to perform accurately. Though there are some publicly available benchmark suites containing millions of labeled images, they are usually restricted to generic objects or urban road scenes and do not contain off-road navigation-relevant classes such as terrain types (gravel, mud, sand, grass, obstacle, etc).
The aim of this project is to overcome this drawback through transfer learning, which involves taking an existing CNN architecture, that is pre-trained for urban road-scene understanding, and investigating a novel fine-tuning architecture for the task of classifying terrain types in off-road scenes, assessing the network performance during the training cycle.
This work is part of the NCCR Digital Fabrication, a large Swiss project involving multiple disciplines that aims to lead the development and integration of digital technologies within the field of architecture. A successful method will directly be used within the semantic mapping pipeline developed for this project and integrated with a robotic excavator for outdoor operation.
- WP1: Familiarization with algorithms and datasets for semantic segmentation.
- WP2: Construction of a small specialized dataset for off-road scene understanding using publicly available imagery.
- WP3: Investigate a new/specialized architecture to fine-tune a state-of-the-art segmentation network for off-road scene understanding.
- WP4: Experimentation with real data and evaluation of results.
- WP1: Familiarization with algorithms and datasets for semantic segmentation. - WP2: Construction of a small specialized dataset for off-road scene understanding using publicly available imagery. - WP3: Investigate a new/specialized architecture to fine-tune a state-of-the-art segmentation network for off-road scene understanding. - WP4: Experimentation with real data and evaluation of results.
- C++/Python programming experience.
- Background knowledge in computer vision and deep learning desired.
- Experience in Linux is beneficial.
- C++/Python programming experience. - Background knowledge in computer vision and deep learning desired. - Experience in Linux is beneficial.
Interested students please send application with CV and transcripts to Ruben Mascaro (rmascaro@ethz.ch) with CC to Zetao Chen (chenze@ethz.ch).
Interested students please send application with CV and transcripts to Ruben Mascaro (rmascaro@ethz.ch) with CC to Zetao Chen (chenze@ethz.ch).