Register now After registration you will be able to apply for this opportunity online.
Action Recognition Using 3D Hand-Object Contact Map
Action recognition is an essential task in computer vision and has numerous applications in various fields, including robotics, surveillance, and healthcare. The recognition of actions involves the analysis of temporal and spatial information within a video sequence. Current state-of-the-art methods use 3D hand and object poses for action recognition, where the object's corners are commonly used for representation. However, this approach has limitations in accurately modeling the hand-object interaction. In [1], we show that leveraging hand-object contact-map representation helps improve action recognition. However, this representation can be learned implicitly for the task of action recognition.
[1] https://arxiv.org/pdf/2309.10001.pdf
Qualifications:
- Experience in Python.
- Interest in Mixed Reality/3D Vision.
- Interest in Machine Learning and Computer Vision.
We aim to achieve this objective through the following tasks:
1. Use existing datasets that have annotated 3D hand and object poses with action labels.
2. Investigate features extracted from RGB and/or 3D Skeletal streams.
3. Investigate different architectures (e.g., transformer) for implicit contact-map prediction.
4. Compare the performance of the proposed method with state-of-the-art methods.
Qualifications:
- Experience in Python.
- Interest in Mixed Reality/3D Vision.
- Interest in Machine Learning and Computer Vision.
We aim to achieve this objective through the following tasks:
1. Use existing datasets that have annotated 3D hand and object poses with action labels.
2. Investigate features extracted from RGB and/or 3D Skeletal streams.
3. Investigate different architectures (e.g., transformer) for implicit contact-map prediction.
4. Compare the performance of the proposed method with state-of-the-art methods.
The primary objective of this project is to use an enhanced representation of 3D hand and object interaction to improve action recognition accuracy
The primary objective of this project is to use an enhanced representation of 3D hand and object interaction to improve action recognition accuracy
Please send an email with your CV and transcript to apply for this opportunity.
Taein Kwon taein.kwon@inf.ethz.ch
Please send an email with your CV and transcript to apply for this opportunity.