SAMPart3D: Segment Any Part in 3D Objects

1 The University of Hong Kong 2 VAST

Paper

Code

Data: PartObjaverse-Tiny

Overview

SAMPart3D, a scalable zero-shot 3D part segmentation framework that segments any 3D object into semantic parts at multiple granularities, without requiring predefined part label sets as text prompts. For scalability, we use text-agnostic vision foundation models to distill a 3D feature extraction backbone, allowing scaling to large unlabeled 3D datasets to learn rich 3D priors. For flexibility, we distill scale-conditioned part-aware 3D features for 3D part segmentation at multiple granularities.

Highlight:

                A pipeline to segment multi-granularity parts in 3D objects.
A 3D point cloud encoder pretrained on large-scale 3D objects from Objaverse, distilling visual features from DINO-v2.
A 3D part segmentation dataset which provides detailed semantic-level and instance-level part annotations for complex 3D objects.

            

Click the thumbnails below to load meshes from Objaverse or Generated meshes from Tripo AI website and Rodin Gen-1 website, and see multi-granularity part segmentation. (The names of the generated meshes are labeled with Tripo-* and Rodin-*.)

Point Clouds Multi-granularity Segmentation

Method

Part Editing

The resulting 3D part segmentation can directly support various applications, including part material editing, part geometry editing, and click-based hierarchical segmentation.

PartObjaverse-Tiny Dataset

PartObjaverse-Tiny, a 3D part segmentation dataset which provides detailed semantic-level and instance-level part annotations for 200 complex 3D objects.

Citation

If you use this work or find it helpful, please consider citing:

@article{yang2024sampart3d,
     author = {Yang, Yunhan and Huang, Yukun and Guo, Yuan-Chen and Lu, Liangjun and Wu, Xiaoyang and Edmund Y., Lam and Cao, Yan-Pei and Liu, Xihui},
     title = {SAMPart3D: Segment Any Part in 3D Objects},
     journal = {arXiv preprint arXiv:2411.07184},
     year = {2024},
    }