Postdoctoral Fellowship in Robot Learning and Perception

Deadline: 08.12.2022


SINTEF er et av Europas største forskningsinstitutt, med flerfaglig spisskompetanse innenfor teknologi, naturvitenskap og samfunnsvitenskap. SINTEF er en uavhengig stiftelse som siden 1950 har skapt innovasjon gjennom utviklings- og forskningsoppdrag for næringsliv og offentlig sektor i inn- og utland. SINTEF har 2100 medarbeidere fra 80 nasjoner og en årlig omsetning på over tre milliarder kroner.

Vår visjon: Teknologi for et bedre samfunn

Les mer om SINTEF her.

SINTEF Ocean er et av seks institutt som utgjør SINTEF. Vi jobber med forskning og innovasjon knyttet til havrommet for nasjonal og internasjonal industri. Vår ambisjon er å videreføre Norges ledende posisjon innenfor marinteknisk og biomarin forskning. Sammen med næringsliv og myndigheter utvikler vi fremtidsrettede løsninger for bærekraftig utnyttelse av havet. Vårt hovedkontor er i Trondheim, og vi har avdelinger i Tromsø, Hirtshals, Oslo, Ålesund og Bergen.

About SINTEF Ocean

SINTEF Ocean is one of the six research institutes that make up SINTEF. SINTEF Ocean conducts research and innovation related to ocean space. Our ambition is to ensure Norway's leading position in marine technology and biomarine research. Many of the challenges of modern society can be solved through sustainable use of the ocean. Transport, food and energy production represent the backbone of ocean-based industries and are also core research areas for SINTEF Ocean. The research focus of the Automation group at SINTEF Ocean is to develop sustainable solutions for the food and seafood processing and production, where automation of food handling and manipulation tasks is one of the fundamental research areas. Therefore, our vision is to develop versatile robot-based solutions that can automate some of the most complex manipulation tasks today involving 3D compliant objects such as seafood.

About the Project

SINTEF Ocean and the Automation group, Trondheim, Norway, have a vacant Postdoctoral fellowship position. The position is available for 24 months. This position is within the SINTEF Ocean-led, ground-breaking FRIPRO/FRINATEK research project – BIFROST, financed by the Research Council of Norway, whose main objective is to develop novel robot visual and tactile-based perception and learning for manipulation of 3D compliant objects. Apart from SINTEF Ocean, who is the project owner, the outstanding project consortium consists of two leading international groups, from MIT CSAIL (USA) and Inria (France).

A major challenge in robotics is the robotic manipulation of the 3D compliant objects. Extending perception (visual and tactile) and learning capability to 3D compliant objects will open a wide range of applications and tasks currently performed only by humans, such as in food production and other areas that are vital to society. The Postdoc fellow will work with embedding the visual (RGB-D) and tactile sensing (GelSight) and using these sensing modalities for learning multi-modal models that can be used for robot perception and learning manipulation tasks on 3D compliant objects.

For the scientific tasks in the project, the Postdoc reports to the PI of the project Dr. Ekrem Misimi, who will serve as the scientific advisor to the Postdoc. The personnel and administrative responsibility of the position lie on the group leader.

Our Group

Our group has an excellent track record in robotic perception and learning for manipulation of compliant objects. Some examples of previous research are related to robot learning by combining deep reinforcement learning (DRL) and Generative Adversarial Networks (GAN) for grasping of compliant objects, or vision-based, robotic manipulation for complex tasks involving compliant objects such as Gribbot. More examples of research in this area can be seen at the webpage of our MAROlab.

About the Position

The Postdoctoral fellow will investigate and develop AI-based learning models by coupling machine learning, computer vision and robotics with the goal of equipping the robots high-level visual and tactile perception capabilities and using these learned models to enable robots with an intelligent and high-level robotic manipulation of 3D compliant objects. This will require a background in AI learning paradigms such as Deep Learning (DL), Reinforcement Learning (RL), Deep Reinforcement Learning (DRL), Learning from Demonstration (LfD) and integration of these paradigms with visual-based sensors (visual and tactile) and robot arm, in simulation and real-world, involving domain adaptation and transfer learning from one domain to another. Therefore, experience in frameworks such as PyTorch or Tensorflow, for AI, OpenCV, PCL or similar for computer vision, and simulation environments such as Nvidia Isaac Gym, MuJoCo or similar, is a prerequisite for the successful candidate.

Example Tasks and Duties

Example tasks with respect to robot perception and learning and some of the duties of the Postdoc candidate are:

  1. Robot perception and learning: To develop a suitable perception and learning framework for multi-modal models for robotic manipulation based on raw visual and image-based tactile observations. The robot perception and learning system should be able to deal with prehensile and non-prehensile manipulation tasks on relevant objects.
  2. AI-based robot learning for shape servoing and active deformation: mapping visual and tactile observations, from the observation space to respective sets of actions, in the action space, to achieve the desired deformation.
  3. Robot learning for switching between the robot perception modes: Develop an occlusion- resilient robot learning framework capable of switching between perception in RGB-D, to perception in RGB, and to perception in image-based tactile only mode.
  4. Publish research in relevant journals and conferences of the field
  5. Co-supervision of MSc-students in the scope of the project

In addition to collaborating with the outstanding consortium partners during the investigation of the tasks above, the Postdoc fellow is expected to collaborate also with the aim of contributing to a final project demonstrator, involving robot perception, and learning for dual-arm manipulation with Panda Franka Emika 7-DOF robotic arms, which is aimed to lead to a final joint project publication.

The basic common robotic platform is composed by a Panda Franka Emika 7-DOF robotic arm(s), which is the arm that will be used during the research work, Intel RealSense D435 RGB-D sensor, GelSight sensor, and Reflex Takktile hand.

Required qualifications

The eligible candidates must have finished a PhD degree in AI, computer science, AI and robotics, cognitive robotics, control, electrical engineering or similar. If you can document that the PhD thesis has been submitted, your application can be evaluated even if you have not yet defended your PhD thesis. Documentation of the obtained PhD degree must be presented before you can take up the position.

The candidate must have the following qualifications:

  • Strong background in AI
  • Strong background in computer vision
  • Excellent programming skills in C++ and Python
  • Fluency in written and oral English
  • Excellent track record - publications in relevant conferences and journals of the field
  • Ability to work independently as well as collaboratively

The following qualification will be considered as an advantage:

  • Experience with simulation environments such as for example Nvidia Isaac Gym, MuJoCo etc.
  • Experience with physical robots

Start date: Preferred start as soon as possible or by the end of Q1 of 2023

Advisor: Ekrem Misimi, senior scientist and PI of the BIFROST project

Working place: SINTEF Ocean - MAROlab, Trondheim, Norway.

About the application:

Application must include the following documents: CV – including publication list, motivation letter, copy of grade transcripts and diplomas of the PhD and MSc studies, and contact information for two references.

In the evaluation of which candidate is best qualified, emphasis will be placed on education, fulfillment of the qualification requirements specified above, experience - track of record, as well as personal qualities and suitability.

Applications sent by email and not by the application form will not be considered.

SINTEF Ocean offers and benefits of working with us:

  • A strong international scientific environment and access to excellent research facilities
  • An open and inclusive work environment
  • A modern pension scheme and excellent insurance
  • Flexible working hours
  • Fully funded position with a highly competitive salary
  • Norway is regularly ranked in the top countries in the world for quality of life

As a Postdoc fellow you will be normally paid at the rate of Postdoc fellows at gross 563 500 NOK annually before tax, but higher rate may be considered based on qualifications and experience of the candidate.

The Postdoc position offers a very good way to get to know to SINTEF and may lead to a permanent position as researcher if both parts are satisfied at the end of the Postdoc position.

If you have any questions about the position, please contact:

If you have any questions about the recruitment process, please contact:

Apply for position


Powered by Labrador CMS