Multi-View Inverse Rendering under Arbitrary Illumination and Albedo
We present the Multi-View Inverse Rendering (MVIR) method that jointly estimates detailed shape, illumination and albedo. Our MVIR uses the initial shape robustly recovered by multi-view stereo and jointly estimates them by using the geometric and photometric smoothness terms and the normalized spherical harmonics illumination model. Our MVIR allows spatially-varying albedo and per image illumination without any prerequisites such as training data or image segmentation.
- MVIR 2016/10/25 for 64-bit Windows 7 and above (a small update on the visibility calculation).
- MVIR 2016/06/01 for 64-bit Windows 7 and above (ECCV 2016 version).
MVIR software requires an initial 3D model (camera poses and 3D mesh). As a basic usage, we create the initial 3D model using VisualSFM and CMPMVS. Please make sure
CMPMVS.exe are on the system path. The script
generate_dataset.bat will then run VisualSFM followed by CMPMVS and re-format their outputs for MVIR.
> generate_dataset [images_folder] [mvir_folder]
[images_folder] is the root of imagery. The
[mvir_folder] will have
mesh.ply as below.
mvir_folder/ ├── images/ │ ├── 00001.jpg │ ├── 00002.jpg │ ├── 00003.jpg │ ├── ... │ └── XXXXX.jpg ├── cameras_v2.txt └── mesh.ply
Now ready to run
> mvirc -i [mvir_folder]
Some control parameters are provided for fine-tuning the MVIR. Use
mvirc --help to see the full list of control parameters.
To test the MVIR software, please first try the tiny dataset The Gajoen 005.
Reproduction the result in 
You can generate the same result by using the Joyful Yell dataset (section 4.2 in ). For this, you need to download our The Joyful Yell dataset and make sure Screened Poisson Surface Reconstruction (Win64,
PoissonRecon.x64.exe) is in your system path, then simply run
> run_joyful [joyful_dataset] [output_folder]
This script automatically perform double MVIR steps with re-computing surface mesh (section 4.2 in ) using control parameters which is same to ours. This process requires ~14GB RAM.
Description of output
The MVIR results are saved in
[mvir_folder]. The most important files are
MVIR-Output_YYYY-MM-DD_HH-MM-SS/ ├── mesh_result.ply ├── mesh_raw.ply ├── illumination.txt │ ├── visibility.txt ├── visibility.ply ├── LightCubeMap_Local/ ├── LightCubeMap_World/ ├── LightSphereMap_Local/ ├── LightSphereMap_World/ └── log.txt
mesh_result.ply: The refined shape (as mesh) and the albedo (as vertex color). The albedo is the one re-computed for improving the visual quality (see section 4.1 in ).
mesh_raw.ply: The raw output from the optimizaton in equation 1 in  (no albedo re-computation).
illumination.txt: The estimated illumination (the spherical harmonics basis).
visibility.txt: The vertex visibility (see section 3.1 in ,
Visibility calculation). The i-th row indicates the camera IDs (0-based) visible from the i-th vertex.
LightSphereMap: The visualization of the estimated illuminations for every input image.
_Localshows the illumination map in the camera coordinates (the same format in our paper ).
_Worldshows the illumination maps w.r.t. the global coordinate system.
log.txt: Processing log of the MVIR software.
The MVIR binary depends on the following third-party software (note that all needed libraries are included in the distribution zip file):
|The Gajoen 005||10 images to test our programs .|
|The Joyful Yell||37 synthetic images, its ground-truth camera parameters, initial mesh and ground-truth mesh.|
Terms and Conditions
MVIR software is provided for research purposes only. In case you use this software for a publication, cite reference , please. See the LICENSE file provided with the demo software.
 Kichang Kim, Akihiko Torii, Masatoshi Okutomi, "Multi-View Inverse Rendering under Arbitrary Illumination and Albedo", In European Conference on Computer Vision 2016. (PDF)