A Skeleton-Driven Neural Occupancy Representation for Articulated Hands


Korrawe Karunratanakul, Adrian Spurr, Zicong Fan, Otmar Hilliges and Siyu Tang


We present Hand ArticuLated Occupancy (HALO), a novel representation of articulated hands that bridges the advantages of 3D keypoints and neural implicit surfaces and can be used in end-to-end trainable architectures. Unlike existing statistical parametric hand models (e.g. MANO), HALO is interpretable and directly leverages 3D joint skeleton defined in Euclidean space as input and produces a neural occupancy volume representing the posed hand surface. The key benefits of HALO are (1) it requires only 3D keypoints as input, which have benefits in terms of accuracy and are easier to learn for neural networks than a set of latent hand-model parameters; (2) it naturally provides a differentiable volumetric occupancy representation of the posed hand; (3) it can be trained end to end, allowing the formulation of losses on the hand surface that benefit the learning of 3D keypoints. We demonstrate the applicability of the HALO model to the task of conditional generation of hands that grasp 3D objects. In this setting, the differentiable nature of HALO is shown to improve the quality of the synthesized hands both in terms of physical plausibility and user preference. The code will be made publicly available for research.

PDF (protected)

  Important Dates

All deadlines are 23:59 Pacific Time (PT). No extensions will be granted.

Paper registration July 23 30, 2021
Paper submission July 30, 2021
Supplementary August 8, 2021
Tutorial submission August 15, 2021
Tutorial notification August 31, 2021
Rebuttal period September 16-22, 2021
Paper notification October 1, 2021
Camera ready October 15, 2021
Demo submission July 30 Nov 15, 2021
Demo notification Oct 1 Nov 19, 2021
Tutorial November 30, 2021
Main conference December 1-3, 2021