🥗 SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulation
ICCV 2023
KAIST
(* denotes equal contribution.)
(a) Shape Generation
Shape A Shape B A→B A→B Refined
(b) Part Mixing Refinement
"A chair with four legs" "rectangle back chair"
(c) Text-guided Part Completion
Abstract
We present a cascaded diffusion model based on a part-level implicit 3D representation. Our model achieves state-of-the-art generation quality and also enables part-level shape editing and manipulation without any additional training in conditional setup. Diffusion models have demonstrated impressive capabilities in data generation as well as zero-shot completion and editing via a guided reverse process. Recent research on 3D diffusion models has focused on improving their generation capabilities with various data representations, while the absence of structural information has limited their capability in completion and editing tasks. We thus propose our novel diffusion model using a part-level implicit representation. To effectively learn diffusion with high-dimensional embedding vectors of parts, we propose a cascaded framework, learning diffusion first on a low-dimensional subspace encoding extrinsic parameters of parts and then on the other high-dimensional subspace encoding intrinsic attributes. In the experiments, we demonstrate the outperformance of our method compared with the previous ones both in generation and part-level completion and manipulation tasks.
HuggingFace Demo
Shape Generation
Part Completion
Part Mixing and Refinement
Text-Guided Shape Generation
Text-Guided Part Completion
More Comparison Results
Comparison of Shape Generation
DPM PVD LION Voxel-GAN Neural Wavelet SPAGHETTI Diff. of $\mathbf{z}$ Diff. of $\{\mathbf{p}_i\}$ Guassians SALAD (Ours)
Comparison of Part Completion
GT Bounding Box Gaussians ShapeFormer Neural Wavelet SALAD (Ours)
Bibtex
@article{Koo:2023Salad,
  title = {{SALAD}: Part-Level Latent Diffusion for 3D Shape Generation and Manipulation},
  author = {Juil Koo and Seungwoo Yoo and Minh Hieu Nguyen and Minhyuk Sung},
  year = {2023},
  journal = {arXiv preprint arXiv:2303.12236}
}