Polarimetric PatchMatch Multi-View Stereo

Abstract

PatchMatch Multi-View Stereo (PatchMatch MVS) is one of the popular MVS approaches, owing to its balanced accuracy and efficiency. In this paper, we propose Polarimetric PatchMatch multi-view Stereo (PolarPMS), which is the first method exploiting polarization cues to PatchMatch MVS. The key of PatchMatch MVS is to generate depth and normal hypotheses, which form local 3D planes and slanted stereo matching windows, and efficiently search for the best hypothesis based on the consistency among multi-view images. In addition to standard photometric consistency, our PolarPMS evaluates polarimetric consistency to assess the validness of a depth and normal hypothesis, motivated by the physical property that the polarimetric information is related to the object's surface normal. Experimental results demonstrate that our PolarPMS can improve the accuracy and the completeness of reconstructed 3D models, especially for texture-less surfaces, compared with state-of-the-art PatchMatch MVS methods.

Our Results

Download Materials

Paper [PDF]
Code [ZIP]
Sample executable file [ZIP]
Sample images [ZIP] (Please refer to Polarimetric MVIR for RAW data.)

Publication

Polarimetric PatchMatch Multi-View Stereo
Jinyu Zhao, Jumpei Oishi, Yusuke Monno, Masatoshi Okutomi
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2024)

Contact

Jinyu Zhao: jzhao[at]ok.sc.e.titech.ac.jp
Jumpei Oishi: joishi[at]ok.sc.e.titech.ac.jp
Yusuke Monno: ymonno[at]ok.sc.e.titech.ac.jp
Masatoshi Okutomi: mxo[at]ctrl.titech.ac.jp