3D Point Cloud Reconstruction

Overview

Reconstructing the 3D world is a fundamental element of many applications including robot manipulation and navigation, scene understanding, view synthesis, virtual reality, and more. Researchers have tried to solve the 3D reconstruction problem using structure from motion and simultaneous localization and mapping. However, the main limitation of these approaches is that they require multiple observations of the desired object or scene from distinct viewpoints with shared features. Such multi-view formulations allow for integrating information from numerous images to compensate for occluded geometry. 3D reconstruction from a single 2D image is a more difficult task since a sole image does not contain the whole topology of the target shape due to self-occlusions. Recent advances in implicit learning allow for reconstructing a target in an arbitrary resolution. To do this, a distance/occupancy field is used to indirectly infer the desired surface. The target surface can then be reconstructed by extracting a zero level set from the distance/occupancy field.

Contributions

  • We created a learning-based implicit model that predicts the unsigned distance between a surface and a query point in 3D space
  • We introduced a state-of-the-art network that implicitly reconstructs a 3D object from a single image without the need for camera parameters during training/inference
  • We further developed our learning-based implicit model by leveraging both raw point cloud data and its discretized voxel counterpart for predicting the unsigned distance between a surface and a query point

Publications

  1. M.S. Arshad and W.J. Beksi, "Automated Reconstruction of 3D Open Surfaces from Sparse Point Clouds," IEEE International Symposium on Mixed and Augmented Reality (ISMAR) Workshops, 2022.
    Paper  •   Preprint  •   Citation
  2. M.S. Arshad and W.J. Beksi, "LIST: Learning Implicitly from Spatial Transformers for Single-View 3D Reconstruction," IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
    Paper  •   Preprint  •   Source Code  •   Citation
  3. M.S. Arshad and W.J. Beksi, "IPVNet: Learning Implicit Point-Voxel Features for Open-Surface 3D Reconstruction," Journal of Visual Communication and Image Representation, 2023.
    Paper  •   Preprint  •   Source Code  •   Citation