Pixelwise Dynamic Convolution Neural Network for LiDAR Depth Data Interpolation

Project Summary

LiDAR data interpolation is challenging because LiDAR data is very sparse and often contains data defect. This project focuses on developing an accurate LiDAR data interpolation method using advanced convolution layer.

Paper Abstract

A light detecting and ranging (LiDAR) system is an important means that takes an omni-directional view to collect precise surrounding 3D information in high sampling frequency. However, due to the architecture of a LiDAR sensor, LiDAR data typically contains much less information in the vertical direction compared to the horizontal direction. In addition, LiDAR data are often defected during measurement. There have been previous researches about LiDAR data interpolation using a high-resolution reference image. However, they suffer from the calibration error due to vibration or shock during operation and do not consider defected data. Here, we propose a reference-free interpolation method with considering the defected LiDAR data using a two-step network architecture. Our proposed method is designed first to exploit the rich information in the horizontal direction of the LiDAR data to restore the defected data and then increase the vertical resolution of LiDAR data. Since the proposed interpolation method is a reference-free approach, it if free from the calibration issue. We confirm that our proposed method shows better performance than existing interpolation methods and is effective as a pre-processing function in other applications. For example, our proposed method can improve the performance of the human segmentation. We also propose a pixelwise dynamic convolution layer as a component of the proposed interpolation method. The proposed dynamic convolution layer adaptively processes each pixel with variable kernels, while existing dynamic layers only use fixed kernels. Experimental results show that the proposed dynamic layer is more effective than existing dynamic layers for LiDAR data interpolation.

Proposed Method Overview

Publications

Pixelwise Dynamic Convolution Neural Network for LiDAR Depth Data Interpolation [Paper]
Wonjik Kim, Masayuki Tanaka, Masatoshi Okutomi, Yoko Sasaki
IEEE Sensors, 2021.