A Large-Scale Multi-Modal Dataset for Sketch-based Engineering Product Modeling

.

Anonymous authors

Tsinghua university

The integration of Artificial Intelligence (AI) and Engineering Product Modeling (EPM) becomes increasingly promising. Freehand sketches and corresponding CAD models are important modalities in data-driven algorithms of sketch-based EPM applications. However, datasets with these paired modalities are rare, and most existing sketch datasets for CAD models either lack a freehand style or are insufficient. Thus, we propose a large-scale multi-modal dataset of paired freehand sketches and CAD models, the SketchEPM dataset. Our dataset covers 23 categories, 9759 CAD models in B-Rep format, 33300 real freehand sketches, and 35873 synthetic sketches. SketchEPM can be easily expanded using our data creation pipeline. We conduct key tasks on our dataset, including sketch recognition, sketch-to-CAD model retrieval, and sketch-based 3D model reconstruction. The experimental results point out the limitations of current models for these tasks. We release the dateset for facilitating the development of sketch-based EPM research and applications.