We present a roughness-aware tensorial representation for robust geometry and material reconstruction from multi-view images. In this scene, we showcase six objects with different materials, including diffuse Lego from TensoIR, specular Horse from NeRO, and several glossy objects from NeILF++ and our datasets, where Qilin and Luckycat are real data. Our method demonstrates robust reconstruction of any reflective objects, detailed geometry results and faithful material estimation, leading to photo-realistic relighting.

Abstract

Reconstructing objects with realistic materials from multi-view images is problematic, since it is highly ill-posed. Although the neural reconstruction approaches have exhibited impressive reconstruction ability, they are designed for objects with specific materials (e.g., diffuse or specular materials). To this end, we propose a novel framework for robust geometry and material reconstruction, where the geometry is expressed with the implicit signed distance field (SDF) encoded by a tensorial representation, namely TensoSDF. At the core of our method is the roughness-aware incorporation of the radiance and reflectance fields, which enables a robust reconstruction of objects with arbitrary reflective materials. Furthermore, the tensorial representation enhances geometry details in the reconstructed surface and reduces the training time. Finally, we estimate the materials using an explicit mesh for efficient intersection computation and an implicit SDF for accurate representation. Consequently, our method can achieve more robust geometry reconstruction, outperform the previous works in terms of relighting quality, and reduce 50% training times and 70% inference time. Codes and datasets are available at https://github.com/Riga2/TensoSDF.

Downloads

BibTex

@article{Li:2024:TensoSDF,
  title={TensoSDF: Roughness-aware Tensorial Representation for Robust Geometry and Material Reconstruction},
  author={Jia Li and Lu Wang and Lei Zhang and Beibei Wang},
  journal ={ACM Transactions on Graphics (Proceedings of SIGGRAPH 2024)},
  year = {2024},
  volume = {43},
  number = {4},
  pages={150:1--13}
)

Back to Homepage