Early Stage Researcher (M/F/D)

in the Marie Skłodowska-Curie Innovative Training Network Project MENELAOS^NT in the field

Deep Depth from Defocus (Deep DFD) for near range and in-situ 3D exploration (ESR05)

Acquiring 3D geometry of the scene is essential for many applications in the areas of navigation, robotics, scene understanding, etc. Among the existing approaches, those using passive devices are of increased interest since they allow the use of compact, standard and low cost imaging systems like DSLR cameras. There are many depth cues that can be used to extract the 3D geometry. In single shot images, the depth is laying in the blur, shadows of objects, chromatic effects and shape distortions caused by lens aberrations, etc.  When multiple images are used, depth information comes from perspective change like in binocular systems or structures motion in video sequences. The physics of these effects is well known and more or less accurate mathematical models exist and are used by analytical image processing methods that are generally prone to heavy calculation.
The entrance of the newcoming Deep Neural Networks (DNN) on the stage of signal processing has boosted the subject due to their capability to learn complex models that ingest multiple effects, not only single ones as analytical approaches are doing. The flexibility in learning and the fast processing once the training is accomplished make DNNs a very promising tool in building the 3D geometry of scenes from easy-to-acquire images.

The successful candidate will be employed for 3 years by University Politehnica of Bucharest and will be part of CEOSpaceTech, a very dynamic research laboratory oriented towards space applications. She or he will receive a financial package, which is twice the average salary in the country and additional mobility and family allowance, granted according to the rules for Early Stage Researchers (ESRs) in the EU Marie Skłodowska-Curie Actions Innovative Training Networks (ITN). A career development plan will be prepared for her/him in accordance with the supervisor. The plan will include a choice of more than 20 streamed or registered courses, stays in Spain, and various outreach activities. For more information, please visit the Marie Skłodowska-Curie Actions Innovative Training Networks website.

YOUR TASKS

• Study of physical foundation for depth cues in images and evaluation of their potential in existing methods for depth mapping.
• Elaboration of DNN-based solutions for depth inference from single shot images by exploiting defocus and other depth cues.
• Definition of benchmarks for DNN training, validation and testing.
•  Evaluation of the accuracy of depth maps obtained with the DNNs using indoor and outdoor image collection.

PROFILE

• A Master of Science in Computer Science is required. It could comprise to the full range of mathematical, physical, engineering and technology disciplines related to sensor data acquisition and programming.
• Proficient English level
• Image analysis, neural networks, programming languages, basic knowledge on optics would be an advantage.

PLANNED SECONDMENTS

• CiTIUS, Santiago de Compostela, Spain, Prof. Dr. P. López Martínez, 3 months, study the possibilities to integrate the coded aperture into the sensor and to understand the impact of sensor technology on the camera performance.
• INSITU, Santiago de Compostela, Spain, Prof. Dr. P. Arias Sánchez, 5 months, refine the camera specification by studying the applications.

ADDITIONAL INFORMATION

References

1. [Gro] Grossmann, Paul. “Depth from focus”. Pattern Recognition Letters 5, no. 1 (1987): 63-69.
2. [Lev] Levin, Anat, Rob Fergus, Frédo Durand, and William T. Freeman. “Image and depth from a conventional camera with a coded aperture”. In ACM Transactions on Graphics (TOG), vol. 26, no. 3, p. 70. ACM, 2007.
3. [Lin] Lin, Jingyu, et al. “Separable coded aperture for depth from a single image.” IEEE Signal Processing Letters 21.12 (2014): 1471-1475.
4. [Zho] Zhou, Changyin, Stephen Lin, and Shree K. Nayar. “Coded aperture pairs for depth from defocus and defocus deblurring.” International journal of computer vision 93.1 (2011): 53-72.
5. [Eig] Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. NIPS (2014)
6. [Liu] Liu, Fayao, Chunhua Shen, and Guosheng Lin. “Deep convolutional neural fields for depth estimation from a single image.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
7. [Car] Carvalho, Marcela, et al. “Deep Depth from Defocus: how can defocus blur improve 3D estimation using dense neural networks?” Proceedings of the European Conference on Computer Vision (ECCV). 2018.
8. [NYU] https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html