Real-Time Force Regulation for Whole-Hand Dexterous Grasping

Preprint  ·  2026

1×
1×
1×

Real-Time Force Regulation for Whole-Hand Dexterous Grasping

Preprint  ·  2026

Sang Min Kim1,†, Alex Alexiev2, Tzu-Yuan Lin2, Sangbae Kim2, Young Min Kim1,*, Yonghyeon Lee3,*
1Seoul National University 2Massachusetts Institute of Technology 3Yonsei University
†Work done while visiting MIT *Co-corresponding authors
Seoul National University 3D Vision Lab MIT Biomimetic Robotics Lab Yonsei University RoPhi Lab
arXiv (Soon) Code (Soon)

TL;DR  Whole-hand force regulation without tactile sensing: contacts on every hand link are estimated from the tracked object model and joint angles, and the contact forces are re-solved every control cycle as the contacts change.

The hand as a contact sensor

About 3,000 points on the hand links are queried against the object's signed distance field each cycle. Points within 1 cm are contacts. Under 1 ms on GPU.

Forces re-solved every cycle

A convex QP allocates contact forces under friction, a minimum squeeze, torque limits and an actuation-consistency constraint. Solved at about 84 Hz.

Reach, grasp, hold, regrasp

With reactive reaching in front, a 27-DoF arm-hand system grasps a moving object, holds it through pushes, and regrasps after losing it.

Abstract

Robust dexterous grasping requires maintaining physical stability despite contacts interactively evolving across the entire hand. A precomputed force distribution can easily fail under object motion, modeling errors, or external disturbances. In this paper, we present a framework for real-time force regulation over dynamically changing whole-hand contacts. Our method geometrically estimates contacts across all hand links using a tracked object model and proprioception, without requiring tactile sensing at those contacts. It repeatedly recomputes the desired contact-force distribution subject to friction constraints, actuator limits, and an actuation-consistency constraint motivated by classical whole-limb force analysis. We integrate this force-regulation controller with reactive reaching, enabling the hand to acquire a grasp, maintain it under disturbances, and regrasp after losing the object. Simulation experiments without gravity demonstrate improved grasp retention over fixed-allocation and fingertip-only execution under controlled perturbations, while real-world experiments on a 27-DoF arm-hand system demonstrate grasp maintenance and recovery under human-applied disturbances as contacts evolve across the whole hand.

Method

Every control cycle: estimate contacts from geometry, re-solve the forces, repeat.

Method overview. A precomputed signed distance field gives contact locations and normals on every link from the joint states and tracked object pose. Contact forces are then allocated for zero net wrench, and both steps repeat in real time.
Snapshots of the hand holding an object under rod perturbations, with commanded contact forces drawn as arrows, and palmar/dorsal hand renderings colored by distance to the nearest contact.
Whole-hand contacts on hardware. Arrows: commanded contact forces while a rod pushes the object. Hands: distance to the nearest contact. Contacts appear on the fingertips, finger sides and the back of the fingers.
1

Whole-hand contact estimation

The object mesh comes from CAD or a short RGB-D scan; its SDF is precomputed once. Each cycle, candidate points on every link are placed with the tracked pose and joint angles, and the closest ones per link become contacts. The palm counts too.

2

Real-time force allocation

A QP minimizes the residual object wrench under friction pyramids, a minimum normal force, joint-torque limits and an actuation-consistency constraint that removes forces the finger joints cannot produce.

3

Integration with reactive reaching

Reactive reaching drives the fingertips and palm to a grasp that follows the tracked object. Thumb plus two fingers in contact hands over to force regulation; if the object is lost, control returns to reaching.

Simulation

MuJoCo, 79 YCB objects, 10 power grasps each, gravity off. Baselines allocate forces once at initialization and hold them as fixed torques (feedforward) or impedance targets.

Perturbation-test protocol: grasp initialization, force regulation, six-axis load test, pose perturbation, release and retest.
Perturbation protocol. Regulate the grasp (a, b), apply a load of 10% of the object weight along ±x, ±y, ±z (c), rotate the object by θ about a random axis (d), and repeat the load test (e).
Success rate binned by number of ground-truth contacts for 790 grasps.
Multi-link contacts. Success over 790 grasps, binned by number of contacts. The gap is largest with 3 to 5 contacts.
Success rate versus target perturbation angle for feedforward, impedance, fingertip-only and ours.
Dynamic perturbation. Ours stays above both baselines at every angle; whole-hand contacts beat fingertip-only, 88.6% vs 72.4% at 90°.
Success rate under synthetic object pose error for three contact thresholds and the feedforward baseline.
Object pose error. With up to 30° and 5 mm of pose offset, feedforward drops to 54% while ours stays at 87% to 93%.

Ablations

Success rate and compute rate versus maximum contacts per link K.
Contacts per link. More points per link means a slower QP: 227 Hz at K=1, 66 Hz at K=7. Success peaks at K=5.
Desired and measured contact forces with and without the actuation-consistency constraint.
Actuation consistency. Without it, a 0.43 N net force is left on the object (red). With it, the net force is nearly zero.

Real world

A 7-DoF Flexiv Rizon 4 arm and a 20-DoF Robotis 5F hand with one RealSense D455 and no tactile sensing. Objects are scanned by hand and tracked at 30 Hz. The three runs play, uncut, in the viewer at the top of the page.

Experimental setup: 27-DoF arm-hand robot, RealSense camera and test objects on a table.
Setup. 27-DoF robot, one RGB-D camera, i9-13900K and RTX 4090.
Full-pipeline runs. Unrecovered losses left the arm workspace.
ObjectDurationPerturbationsLossesRecovered
Pringles59 s1633 / 3
Small box48 s3121 / 2
Large box94 s3132 / 3

78 perturbations over about 200 s; six of eight losses recovered without intervention.

Beyond the reach of a grasp taxonomy

Tactile skins sense only where they are mounted, and grasp priors place contacts where a taxonomy expects them. Our estimator searches the whole hand surface, so neither assumption is needed: here the fingers alternate in front of and behind the can, and the ones holding it with their dorsal surface are regulated through the same contact Jacobian.

A grasp beyond the taxonomy. Right: palmar and dorsal views colored by distance to the object. The fingers behind the can register contact on their dorsal surface, and the grasp holds while a rod pushes it.

BibTeX

TBA