RealisticHands: A Hybrid Model for 3D Hand Reconstruction


Michael Seeber, Roi Poranne, Marc Pollefeys and Martin R. Oswald


Estimating 3D hand meshes from RGB images robustlyis a highly desirable task, made challenging due to the nu-merous degrees of freedom, and issues such as self simi-larity and occlusions. Previous methods are generally di-vided to parametric 3D hand models and model free ap-proaches. While the former can be considered more ro-bust, e.g. to occlusions, they are less expressive. We pro-pose a hybrid approach, utilizing deep neural network anddifferential rendering based optimization to demonstrablyachieve the best of both worlds. In addition, we explore Vir-tual Reality (VR) as an application. Most VR headsets arenowadays equipped with multiple cameras, which we canleverage by extending our method to the egocentric stereodomain. This extension proves to be more resilient to theabove mentioned issues. Finally, as a use-case, we showthat the improved image-model alignment can be used toacquire the user’s hand texture, which can be for a morefaithful virtual hand representation.

PDF (protected)

  Important Dates

All deadlines are 23:59 Pacific Time (PT). No extensions will be granted.

Paper registration July 23 30, 2021
Paper submission July 30, 2021
Supplementary August 8, 2021
Tutorial submission August 15, 2021
Tutorial notification August 31, 2021
Rebuttal period September 16-22, 2021
Paper notification October 1, 2021
Camera ready October 15, 2021
Demo submission July 30 Nov 15, 2021
Demo notification Oct 1 Nov 19, 2021
Tutorial November 30, 2021
Main conference December 1-3, 2021