Hardik Jain and Olaf Hellwich

In the past few decades, the computer vision domain has achieved outstanding success in learning 3D shapes for classification, segmentation and image-based reconstruction. However, deep networks are less explored for the generative task of obtaining new 3D shapes from the learned representation. This problem becomes more prominent for 3D shapes represented as surface meshes, mainly because the mesh structure lacks regularity, an essential property for training deep generative networks. In this work, we remedy this problem by proposing a generative icosahedral mesh convolutional network (GenIcoNet) that learns data distribution of surface meshes. Our end-to-end trainable network learns semantic representations using 2D convolutional filters on the regularized icosahedral meshes. During inference, GenIcoNet can be used to generate new geometrically valid shapes directly as surface meshes. Our experiments for interpolation of latent space demonstrate that GenIcoNet is able to outperform networks trained on intermediate surface mesh representations. The variational autoencoder architecture of GenIcoNet learns meaningful representation which is numerically stable w.r.t. small perturbations, allows performing exploration and combination of surface meshes to generate new meaningful shapes, while maintaining the essential property of mesh manifoldness.