Skip to content

SouthwardTech

Technology, industry and investment across China and the Global South

5 projects tracked1 Contracted4 AnnouncedReviewed

China

How Nvidia used sensors from Beijing to teach a robot to open a jar

Ten fingertip sensors made by Tashan recorded what a camera cannot: the squeeze of a human hand. They are now on the parts list for the device Nvidia uses to gather training data for robots.

A person wearing Nvidia’s DexUMI frames handles a jar of granola.
A person wearing Nvidia’s DexUMI frames handles a jar of granola. Image: Nvidia

Next time you open a jar, pay attention to the hand holding it still. How hard are you squeezing?

You probably cannot say. Neither can a camera, which is a problem for anyone teaching a robot by filming people at work. So when Nvidia’s researchers recorded a person opening a jar of granola, they fitted a jointed frame over each hand. As the lid turned, a sensor at every fingertip measured the force. All ten came from Tashan, a Beijing company, according to Nvidia’s parts list.

The frame, called DexUMI, was used to collect data for GR00T 2, the robotics model Nvidia introduced at the Conference on Robot Learning in Austin, Texas, on 9 November. Its tasks included making breakfast. Pictures published by the researchers show a person slicing a banana and scooping granola into a bowl, then a pair of robot arms taking a turn.

Nvidia has published DexUMI’s design so other labs can build their own. That makes Tashan’s sensor a listed part in a tool built to gather one of the scarcest things in robotics: recordings of skilled hands at work. Tashan told SouthwardTech it was Nvidia’s sole partner for the touch sensors on DexUMI’s fingertips and palms. It said it was offering the same fingertip sensor as open-source hardware.

The missing textbook

Chatbots learned to write from text already on the internet. Ken Goldberg, a roboticist at the University of California, Berkeley, has estimated that a person would need about 100,000 years to read it all. No such library exists for how a hand moves or how tightly it grips. Every example has to be recorded.

The usual method is teleoperation: a person steers a robot through a task. It is slow. Eight hours of work produces eight hours of data, Goldberg notes. And the operator feels nothing the robot touches. A frame like DexUMI lets a person do the task directly, with no robot in the room. In a short test in 2025, its original designers found it gathered demonstrations about three times as fast as teleoperation.

Nvidia also trained GR00T 2 on 100,000 hours of first-person footage. But footage shows where fingers go. It cannot show how hard they press.

What the fingertips recorded

That is the job of Tashan’s sensors. Each measures two things: how firmly a fingertip presses against an object, and how much force it exerts along the surface as it twists. The frame also kept a record of how each finger bent. Together, the two gave the researchers a way to teach the robot how much to yield when its fingers met resistance.

What each part of the DexUMI frame records.
What each part of the DexUMI frame records. Graphic: SouthwardTech; source: Nvidia

The original DexUMI carried whichever touch sensor came with the robot hand it was built for, and its authors reported that readings drifted under a human’s stronger grip. Nvidia’s version specifies one Tashan model for all ten fingertips.

Opening the jar was only the first step. The robot also had to pick up a spatula, scoop granola into the bowl, pass the jar back to its other hand and set it down. Nvidia reports that it finished the whole sequence in 80 per cent of trials. Two other models trained on demonstrations of the same task never did, it says.

A name on the parts list

Nvidia’s parts list gives builders a shopping link for Tashan’s ten fingertip sensors. It also lists a Tashan sensor for each palm, though the assembly guide says those were not used to train the model and can be swapped for 3D-printed parts.

The sensors and electronics on Nvidia’s parts list for DexUMI.
The sensors and electronics on Nvidia’s parts list for DexUMI. Graphic: SouthwardTech; source: Nvidia

The relationship predates the hardware. Tashan told SouthwardTech it was the world’s first partner for simulating touch in Isaac Sim, Nvidia’s robotics simulator. A year earlier, it had given the simulator’s users a virtual touch sensor, which Nvidia’s sensor catalogue lists. Tashan’s instructions describe dropping a card onto the sensor and watching the readings change. That sensor is a different product from the ones on DexUMI.

Tashan has a second stake in how robots learn. In September, the company and computer scientist Richard Sutton opened a “robot kindergarten” in Beijing to study how machines might learn from their own mistakes.

A different pair of hands

Nvidia’s own tests show where the readings stop helping. In a package-sorting experiment, a person wore the frames to show a robot how to pick packages off a moving belt and toss them into a bin.

This robot’s hand had different finger joints. The researchers could still work out where its fingertips should go, but the same push at a fingertip put a different load on the joints behind it. So they trained the model on finger positions and set aside the force calculations from the original hand.

The person had shown the new hand where to reach for the package. How hard should it squeeze when it got there? The sensors had measured a human grip. Those readings alone could not tell a differently shaped hand how to reproduce it.

Published 11 November 2026 · Last updated 11 November 2026 · Corrections to corrections@southwardtech.com

Weekly newsletter

The Southward Brief

A weekly briefing on the technologies, companies and industrial shifts reshaping China and the Global South.

Required. Enter a valid email address.

By subscribing, you agree to receive The Southward Brief. You can unsubscribe at any time. Read our Privacy Policy.