A new open-source model for hand estimation from images has been released and it is fast on consumer hardware: FAH. The paper appeared on arXiv on 5 November 2026 and the code is on GitHub; the author is Jinkun Cao. From a normal image or video, the model predicts 2D and 3D points of the joints and the surface of the hand, and fits a MANO or MHR-Hand mesh to them.
What FAH does
Most hand mesh estimators predict rig parameters and obtain 2D points by projecting the mesh. The author says that in his experiments directly predicted 3D joints agree better with the 2D predictions, and fitting a mesh to the predicted points reduces mesh error. FAH is built on the hand branch of SAM 3D Body (through fine-tuning and feature distillation) and provides:
2D and 3D joints and surface points, plus a MANO or MHR-Hand fit
Optional: hand-part labels and object-contact probabilities, with no object geometry needed at inference
Video mode: causal refinement (past frames only) that lowers acceleration error
World mode: camera estimation that places the hands in world space
FAH-L outputs on a held-out HOT3D clip: joints, surface points, mesh, hand parts and masks.
Four sizes, with speed and accuracy
The FAH family has four models. The table shows the numbers from the repository README; PA-MPJPE is the joint error (lower is better) on HOT3D, and speed is measured on an RTX 4070 Ti SUPER with TensorRT and mesh fitting.
Model | Encoder | Crop size | PA-MPJPE (mm) | Speed (fps) |
|---|---|---|---|---|
FAH-S | DINOv3 ViT-S/16 | 320 | 5.25 | 101 |
FAH-B | DINOv3 ViT-B/16 | 320 | 4.57 | 89 |
FAH-L (default) | DINOv3 ViT-L/16 | 320 | 4.31 | 65 |
FAH-H | DINOv3 ViT-H+/16 | 448 | 4.12 | 27 |
With hand-part labels FAH-L drops to 54 fps, and with contact as well to 49 fps. FAH-S reaches 125 fps without mesh fitting, parts and contact. According to the paper, FAH-L is about 6.5 times faster than the hand branch of SAM 3D Body (65 vs 10 fps).
desk work with a camera, a toy, a mouse and a keyboard, with all outputs (input, 2D keypoints, 3D mesh, parts, contact).
gestures, overlapping hands and handling an object facing the camera.
The paper
The paper is titled "Fast and Accurate Hand Estimation in Pixels, Interaction and World Space". According to the paper, on HOT3D the video mode lowers acceleration error from 3.83 to 1.65 (HaWoR, the video baseline it compares with: 1.94) and PA-MPJPE from 4.31 to 4.16 mm. In 2D accuracy FAH-L reaches 0.83 PCK@0.1 against 0.72 for the strongest baseline. On contact it beats HACO according to the paper (HOT3D with 10 mm labels: F1 0.80 vs 0.58).
According to the project page, HACO flags contact throughout these videos, while FAH-L flags 69–95% of object holds.
The paper's limitations say that camera error dominates the error in world space. The video gain is also not the same on every metric: on EgoStandard acceleration error falls from 3.99 to 1.67 but PA-MPJPE gets slightly worse (7.10 to 7.17 mm). The README adds that tiny hands in collages and tightly crossed hands remain difficult, and that contact can also fire on hand-face or hand-hand contact.
A head-mounted video sample from the HoloAssist dataset, trimmed and overlaid with the model's predictions.
Full paper (PDF): Fast and Accurate Hand Estimation in Pixels, Interaction and World Space
Install and license
According to the README you need Python 3.10 and PyTorch 2.1 or newer. The command fah --img hand.jpg --output_dir output/fah runs on one image and fah-video is provided for video. Installation on CPU and Apple silicon is described too, but the reported speeds were measured only on an NVIDIA GPU. Some practical points:
The code is Apache-2.0, but all FAH weights are for research and non-commercial use only.
To run it you must accept the SAM license on Hugging Face, and register on mano.is.tue.mpg.de to download MANO separately.
My take
It seems the main appeal of FAH is the mix of speed and many outputs: judging from the published specifications, about 65 fps with mesh fitting on one consumer card is something that can matter for hand tracking in AR/VR interfaces, robotics and data capture. On the other hand, all numbers come from one author and one setup (WSL2 on an RTX 4070 Ti SUPER), and the non-commercial license of the weights limits use in a product. If I were to test it, I would first run FAH-S and FAH-L on the same video and measure the real speed and the stability on hands holding objects.
