MeshCoder

LLM-Powered Structured Mesh Code Generation from Point Clouds


1Shanghai AI Lab    2Tsinghua University    3Harbin Institute of Technology   
4Beijing Institute of Technology    5HKUST(GZ)   

*Equal contribution    Corresponding author

MeshCoder Project Overview

MeshCoder generates structured mesh code from 3D point clouds, enhancing shape editing and understanding.

Abstract

Pipeline

Reconstructing 3D objects into editable programs is pivotal for applications like reverse engineering and shape editing. However, existing methods often rely on limited domain-specific languages (DSLs) and small-scale datasets, restricting their ability to model complex geometries and structures.

To address these challenges, we introduce MeshCoder, a novel framework that reconstructs complex 3D objects from point clouds into editable Blender Python scripts. We develop a comprehensive set of expressive Blender Python APIs capable of synthesizing intricate geometries. Leveraging these APIs, we construct a large-scale paired object-code dataset, where the code for each object is decomposed into distinct semantic parts.

Subsequently, we train a multimodal large language model (LLM) that translates 3D point clouds into executable Blender Python scripts. Our approach not only achieves superior performance in shape-to-code reconstruction tasks but also facilitates intuitive geometric and topological editing through convenient code modifications. Furthermore, our code-based representation enhances the reasoning capabilities of LLMs in 3D shape understanding tasks. Together, these contributions establish MeshCoder as a powerful and flexible solution for programmatic 3D shape reconstruction and understanding.

Point Cloud Input

Code-generated mesh(with wireframe)

MeshCoder generate mesh code from point cloud

run code in 3d editing softwares (blender) and generate high quality mesh

Generated Code

Gallery

3D Mesh

Tip

Loading 3D Model...

Editable Code

Tip

Shape Editing

From point clouds to part-level reconstructed meshes, MeshCoder now enables detailed shape editing by modifying the generated code.

Shape Editing Type Shape Editing Resolution

Shape Understanding

We can also use the generated code to help LLMs better understand the shape.

Shape Understanding Process

Generated Sofa Code

GPT-4o Icon

Generated code could help LLMs better understand the shape

Shape Analysis

Q: What is the structure of this object?
Q: How are the armrests constructed?
Q: How many legs does this sofa have?

BibTeX

@article{MeshCoder,
  author    = {Bingquan Dai and Li Ray Luo and Qihong Tang and Jie Wang and Xinyu Lian and Hao Xu and Minghan Qin and Xudong Xu and Bo Dai and Haoqian Wang and Zhaoyang Lyu and Jiangmiao Pang},
  title     = {MeshCoder: LLM-based Mesh Code Generation},
  journal   = {NeurIPS},
  year      = {2025},
}