Congratulations to our ESR 3 from MENELAOSNT, Faisal Ahmed, who successfully defended his PhD thesis on the topic of “Pseudo-passive Indoor ToF 3D Sensing Exploiting Light-based Wireless Communications Infrastructure” on December 17, 2024 at the Universität Siegen – Center for Sensor Systems (ZESS), Germany under the supervision of Dr. Miguel Heredia Conde and Prof. Paula López Martínez.
3D Time-of-Flight (ToF) cameras have recently aroused a lot of attention from academic and indus-trial circles due to their wide range of applications, such as robot navigation, indoor 3D sensing, autonomous driving, and industrial 3D perception and localization. Despite relevant advancements in ToF imaging, state-of-the-art ToF cameras still suffer from concomitant challenges, e. g., power-hungry illumination sources, interference from existing lighting infrastructure leading to background noise, and measurement inconsistencies caused by thermal drift from the integrated source. Hence, this thesis aims to address these key challenges and propose novel methods and hardware solutions. Our work is inspired by the recently introduced visible light communication (VLC) or light-fidelity (Li-Fi) infrastructure to use existing VLC or LiFi nodes as opportunity illuminators for ToF imaging, instead of the integrated source. VLC/LiFi sources already serve as communication nodes in indoor settings. Recently, the VLC infrastructure has been laying a solid foundation for passive sensing.
This work aims to unveil the synergistic potential of two independently developed fields: optical wireless communication (OWC) and ToF imaging, provided by their interaction. Our general concept is encouraged by passive radar technology, which uses communication infrastructure for target sensing, to achieve passive ToF operation. Exploiting a non-co-located opportunity illuminator induces a passive sensing setup in this thesis. For this purpose, we introduce mathematical machinery that leads to merging the communication and ToF sensing models into a unified imaging pipeline. The passive modality builds upon the concept of coded ToF, yielding pulse-shaped correlation sampling and is implemented leveraging the Photonic Mixer Device (PMD) technology, initially developed at ZESS, which is known as a reference technology in the field of ToF sensors, to attain depth and intensity profiling of targets. Such capabilities are achieved by exploiting the pulse shape of optical signals generated by VLC/Li-Fi modules (e. g., OpenVLC and LiFiMAX). Within this scope, first, we propose an asynchronous passive approach, which uses the communication signals for illumination, and the reference signals are generated internally. This configuration is sensitive to frequency mismatch, which results in an unknown offset. To target this problem, we use two parallel sensing channels. A direct channel is established between the OWC source and a reference photodetector, which is responsible for obtaining a stable reference signal for synchronizing the ToF camera externally. The indirect channel is given by the reflections of illumination signals from the target, resulting in a bistatic configuration. In contrast, conventional ToF imaging uses a monostatic configuration, where a single sensing channel is exploited and the camera is synchronized through internally generated signals.
Furthermore, we also studied different sampling schemes in the time-shift domain, including uniform, random, and sparse rulers, which preserve high-depth accuracy with a minimal number of measurements. On the hardware front, we developed a custom-designed solution to obtain the thresholded version of the VLC optical signals. This solution controls the demodulation of the ToF evaluation board. On the computational front, we adapted the image formation model to the bistatic configuration and proposed algorithms to recover depth maps from samples of the cross-correlation between emitted and demodulation signals. Additionally, we evaluated the performance of cross-correlation functions of well-known modulation schemes in the communication community and further used them for depth reconstruction analysis of the proposed method, leveraging existing transient imaging datasets. Experimental evaluation of the proposed passive ToF camera has confirmed the potential of attaining relevant power savings, e. g., up to 59 % for the Infineon ToF sensor IRS2381C, which features a 1W illumination source.
When the source location is known, reconstructing a target’s depth map can be carried out leveraging existing methods for pulsed ToF imaging. However, the fundamental challenge with passive ToF imaging is the unknown source location, which precludes the estimation of absolute distances. This dissertation proposes novel computational methods to solve this problem. Our solutions are motivated by the Manhattan world assumption, which allows us to consider the target as locally plain. The proposed blind source localization algorithm uses a gradient descent approach for jointly estimating the source location and retrieving the correct depth information of the scene. In each iteration, the bistatic configuration and plane-fitting work as a basic framework while seeking local planarity to constrain the source location. The previous approach has been shown to attain satisfactory results from blind initial guesses of the source location. We also developed a probabilistic approach based on the Kalman filter (KF) for keeping track of the source location across frames, yielding a robust initialization for the source localization algorithm. Different motion models are considered to evaluate the performance of KF-driven algorithm. The numerical experiments performed using emulated measurements of the ToF camera corroborate our proposed algorithms. Indoor 3D sensing, autonomous driving, and industrial 3D perception and localization are rapidly growing fields where VLC infrastructure is often found, showcasing the applications of this work.
Recent Comments