Princeton University

School of Engineering & Applied Science

Sensing, Imaging, and Deep-Learning at Speed in Robotic Swarms of UAVs using 4D Information Theory, mm-wave Radar, and Computer Vision

Jose Martinez-Lorenzo, Northeastern University
B205 Engineering Quadrangle
Wednesday, December 12, 2018 - 4:30pm
mm-wave sensing and imaging systems are ubiquitously used in a wide range of applications, such as atmospheric sounding of the earth to forecast the weather, non-destructive-testing to assess the condition of civil infrastructures, and security monitoring to detect potential threats in airport checkpoints. These systems typically operate well when the scene dynamics does not change rapidly. Unfortunately, this is not the case in emerging societally-important applications like swarms of drones in rescue missions, smart self-driving cars on roadways, or cyber-physical systems searching for suicide bombers when they are on the move. These new applications require sensing and imaging at high video frame, as well as adapting the sensing process to the outcomes of the online learning. With these challenges in mind, one of the key features of the next generation imaging systems will be the ability maximize the information transfer efficiency between the sensors and the imaging domain. One way to achieve this is to merge traditional 1D temporal mm-wave coding with 3D dynamical coding of the wavefield in space, as well as to dynamically adapt the sensing process based on the information provided by other orthogonal sensors, like 4D stereo cameras. This talk will cover the theoretical principles and fundamental limitations of new sensing and imaging systems that leverage on Information Theory to enable 4D coding at mm-wave frequencies–by using spatial modulators, vortex-meta-lenses, and compressive reflectors. The combination of this 4D-coding scheme with compressive sensing techniques reduce the number of sensors needed to produce images with high spatial resolution; and the use of a new Alternating Direction Method of Multipliers (ADMM) algorithm, which is capable of imposing linear combinations of regularizers of the form p-norm raised to the q-power, enables quasi-real-time imaging in a distributed fashion. A Deep-Learning approach that enables the fusion of mm-wave radar and 4D stereo cameras will be discussed as a mechanism to enhance the sensing process even more; and preliminary results of these principles will be presented for two of our experimental prototypes: 1) a cyber-physical imaging system used for passenger screening applications, and 2) a swarm of robotic drones used on rescue missions.
Jose Martinez-Lorenzo is an Assistant Professor at Northeastern University, Boston, MA; where he acts as the Director of the Sensing, Imaging, Control, and Actuation Laboratory. He holds a joint appointment in the Departments of Electrical and Computer Engineering and Mechanical and Industrial Engineering. Prof. Martinez-Lorenzo’s research focus is on high-capacity sensing and imaging systems, with an emphasis on computational modeling of acoustic, thermoacoustics, and electromagnetic waves through complex media, including electromagnetic and acoustic metamaterials and sensors; physics-based signal processing and machine learning, including compressive sensing and information-based imaging and optimization; microwave, millimeter wave and acoustic mechatronics system design and integration; and high-performance computing in multi-core architectures. This fundamental research is applied to diverse problems, including but not limited to civil infrastructure assessment using swarms of drones and deep-learning radar, environmental sensing using distributed sensor networks, and “on-the-move” detection of explosives-related threats. He received a 2017 NSF Early CAREER Award, for the research program entitled 4D mm-Wave Compressive Sensing and Imaging at One Thousand Volumetric Frames per Second. His work on computational modeling, sensors, compressive sensing, and machine learning has received four Best Paper Awards in conferences with over 1200 papers. He has published over 70 journal papers and 130 conference papers, and his work has been featured by CNN, NBC, New York Times, and Wired Magazine.