TacTalentIs contact-rich manipulation still a moat for specialist policies?
Studies including GPT-as-Policy, RoboProbe, RoboDawn and RoboICL[1–7] explore general multimodal models as robot policies: models generate actions from observations, instructions and demonstrations, then revise their decisions through interaction. This offers an alternative to task-specific policy training and raises a fundamental question: what underpins the advantages of specialist models?
Contact-rich manipulation is an important candidate. Precise insertion, stable grasping and material discrimination require more than recognizing objects and their positions: they involve physical information such as contact geometry, slip and material properties. Touch is therefore an important source of sensory information for these tasks. Can general models directly use visual and tactile information to perform contact-rich manipulation, or do these tasks still require specially trained policies?
We investigate this question on UniVTAC[8] and OpenViTac[9]. Using RoboICL[4], we provide GPT-6 Astra with a few demonstrations and compare vision-only and visuo-tactile inputs. We ask two questions: can Astra directly use vision-based tactile images to perform contact-rich manipulation? How well can it perform using vision alone?
A general model in the control loop
We evaluate Astra on nineteen simulated tasks from UniVTAC[8] and OpenViTac[9]. Adopted by multiple tactile-policy studies, UniVTAC offers an established evaluation setting with accumulated baselines across eight tasks spanning grasping, placement, classification and precise insertion. The recently introduced OpenViTac includes eleven tasks that further cover physical-property discrimination and precise contact manipulation; its corresponding simulated and real-world tasks, together with the strong correlation between their evaluation results, support the quality of its evaluation framework. We use a 10% stratified sample with sampling seed 42: the sample includes 80 UniVTAC and 130 OpenViTac episodes, including 30 episodes across the three Insert Block v2 shapes.
Our control pipeline builds on RoboICL[4], which organizes demonstrations and online interaction history into observation–action–outcome–observation sequences. This allows a general-purpose model to follow demonstrations and use execution feedback for closed-loop control. We use GPT-6 Astra with xhigh reasoning effort, without training a task-specific policy.
One-shot tasks use 12 segments from a single demonstration. Tasks with distinct material classes or decision branches use two demonstrations with five segments each; Insert Block v2 uses demonstrations of three shapes. Segment length H is selected with reference to task trajectory length and held fixed within each task: 5 by default, 10 for Lift Bottle, Put Bottle, Empty-Can Selection, Grasp Chip, Gear Assembly and Insert Block v1 and v2, and 4 for Grasp Classify. The selected demonstrations remain available throughout the episode.
Within this framework, we compare vision-only (V) and visuo-tactile (V+T) input. V uses visual images alone, while V+T additionally provides left and right tactile RGB images; both conditions use the same demonstrations and evaluation scenes. Camera selection for each task strictly follows the original benchmark’s official configuration. Alongside images, Astra receives robot proprioception and task text, and outputs seven-dimensional actions specifying end-effector translation, rotation and a gripper target.
Some original Grasp Chip success conditions impose unnecessarily strict constraints on execution details. We therefore simplify these conditions: an episode succeeds when the chip remains intact and is grasped, transferred and placed in the target region. All other tasks retain the official success criteria.
Overall performance and visuo-tactile capabilities
3.1 Astra outperforms specialist policies by a wide margin on contact-rich tasks
With visuo-tactile input, Astra achieves mean success rates of 93.8% on UniVTAC and 81.8% on OpenViTac. These exceed the strongest reported baselines in our tables—SimpleTouch at 70.9% and OpenVTLA at 68.7%—by 22.9 and 13.1 percentage points. Vision-only Astra reaches 76.3% and 80.6%, also exceeding the highest baseline mean in each benchmark. Contact-rich manipulation therefore does not constitute an exclusive domain of capability for specialist models.
| Model | Lift Bottle | Pull-out Key | Lift Can | Put Bottle | Insert Hole | Insert HDMI | Insert Tube | Grasp Classify | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| ACT | 42 | 28 | 20 | 28 | 19 | 15 | 45 | 50 | 30.9 |
| VITaL | 72 | 47 | 8 | 32 | 25 | 6 | 34 | 100 | 40.5 |
| π₀.₅ | 92 | 36 | 59 | 31 | 26 | 3 | 34 | 48 | 41.1 |
| FTP-pi0.5 | 77 | 30 | 26 | 19 | 47 | 4 | 72 | 97 | 46.5 |
| UniVTAC-ACT | 71 | 46 | 29 | 31 | 24 | 28 | 56 | 99 | 48.0 |
| FTP-1 | 97 | 48 | 65 | 47 | 64 | 3 | 79 | 100 | 62.9 |
| SimpleTouch | 100 | 62 | 68 | 50 | 85 | 3 | 100 | 99 | 70.9 |
| Astra (V) | 100 | 30 | 80 | 90 | 60 | 50 | 100 | 100 | 76.3 |
| Astra (V+T) | 100 | 90 | 100 | 90 | 100 | 80 | 90 | 100 | 93.8 |
| Model | Weight Classify | Hardness Classify | Roughness Classify | Roughness-Guided Regrasp | Empty-Can Selection | Grasp Chip | Gear Assembly | Pull Drawer | Insert USB | Insert Block v1 | Insert Block v2 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| StarVLA (Qwen3) | 44 | 38 | 42 | 35 | 32 | 10 | 26 | 35 | 4 | 0 | 0 | 24.2 |
| StarVLA (Cosmos) | 51 | 67 | 94 | 9 | 13 | 1 | 33 | 22 | 3 | 1 | 3.3 | 27.0 |
| Lingbot-VA | 50 | 44 | 39 | 40 | 44 | 22 | 59 | 15 | 2 | 2 | 6 | 29.4 |
| FastWAM | 49 | 84 | 67 | 7 | 9 | 50 | 51 | 88 | 6 | 0 | 0 | 37.4 |
| XVLA | 28 | 30 | 56 | 47 | 45 | 42 | 84 | 76 | 2 | 3 | 0.7 | 37.6 |
| Xiaomi-Robotics | 52 | 46 | 54 | 62 | 38 | 24 | 73 | 84 | 5 | 2 | 0 | 40.0 |
| Lingbot-VLA-2 | 52 | 58 | 58 | 53 | 52 | 41 | 78 | 92 | 8 | 5 | 2.7 | 45.4 |
| π₀.₅ | 53 | 60 | 64 | 52 | 48 | 39 | 87 | 99 | 14 | 9 | 9.3 | 48.6 |
| InternVLA-A1.5 | 53 | 76 | 65 | 77 | 47 | 53 | 79 | 98 | 3 | 6 | 9.3 | 51.5 |
| FTP-1 | 61 | 100 | 100 | 78 | 50 | 60 | 83 | 94 | 27 | 12 | 8.3 | 61.2 |
| OpenVTLA | 89 | 99 | 100 | 90 | 73 | 69 | 91 | 99 | 27 | 11 | 8 | 68.7 |
| Astra (V) | 50 | 70 | 100 | 100 | 70 | 100 | 80 | 100 | 100 | 40 | 76.7 | 80.6 |
| Astra (V+T) | 40 | 90 | 100 | 100 | 50 | 100 | 80 | 100 | 100 | 70 | 70 | 81.8 |
Beyond aggregate success rates, Astra exhibits striking behaviors during execution. The following eight visuo-tactile examples illustrate tactile inference, active probing, fine adjustment and manipulation recovery. Red text highlights tactile cues in the action notes.
Tactile inference
Pull-out Key
ACTION NOTE
COMMAND
Insert Hole
ACTION NOTE
COMMAND
Active probing
Hardness Classify
ACTION NOTE
COMMAND
Fine adjustment
Grasp Chip
ACTION NOTE
COMMAND
Gear Assembly
ACTION NOTE
COMMAND
Insert USB
ACTION NOTE
COMMAND
Insert Block v1
ACTION NOTE
COMMAND
Manipulation recovery
Insert Block v2
ACTION NOTE
COMMAND
3.2 Astra demonstrates strong capabilities in interpreting and using tactile information
UniVTAC
OpenViTac
As shown in Figure 2, adding touch improves success rates on six tasks. On UniVTAC, Pull-out Key rises from 30% to 90%, Lift Can from 80% to 100%, Insert Hole from 60% to 100%, and Insert HDMI from 50% to 80%. On OpenViTac, Hardness Classify improves from 70% to 90%, and Insert Block v1 from 40% to 70%. Spanning contact constraints, grasping, precise insertion and material-property judgments, these gains show that Astra can effectively use tactile information in few-shot control without additional tactile-policy training. However, vision-only and visuo-tactile inputs yield equal success rates on several tasks, including Grasp Classify, Roughness Classify and Roughness-Guided Regrasp. Adding touch even reduces success rates on Weight Classify, Empty-Can Selection, Insert Tube and Insert Block v2. Section 3.3 examines possible explanations for these observations.
The videos below compare vision-only and visuo-tactile performance on the six tasks whose success rates improve with touch. Action notes related to touch are highlighted in red to illustrate how Astra uses tactile information to guide manipulation.
Pull-out Key
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
Lift Can
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
Insert Hole
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
Insert HDMI
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
Hardness Classify
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
Insert Block v1
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
3.3 Astra’s visual perception can partially compensate for missing tactile information
The two benchmarks show different tactile gains: adding touch improves mean success by 17.5 percentage points on UniVTAC, but by 1.2 points on OpenViTac. This difference may reflect their respective design priorities. UniVTAC emphasizes evaluating the utility of touch and restricts certain visual cues to reduce opportunities for solving tasks through vision alone; most tasks, for example, omit the wrist camera. OpenViTac places greater emphasis on alignment with real-world manipulation, providing both fixed and wrist cameras across all tasks and preserving richer visual information. For Astra, these cues may partially compensate for missing tactile information and support judgments otherwise associated with touch, reducing the additional benefit of tactile input.
Grasp Classify, Roughness Classify and Roughness-Guided Regrasp illustrate this effect: vision-only success is 100% on all three. Audited Grasp Classify frames show visible surface texture on the rough cylinder. In Roughness Classify, the wrist camera reveals ridges and a jagged silhouette on rough objects. In Roughness-Guided Regrasp, surface ridges are discernible even from the fixed camera, allowing Astra to select the rough object before contact. The policy’s action notes also refer to these visual differences.
Astra demonstrates visual discrimination not evident in previously reported specialist-model results, uncovering subtle texture, contour and motion cues that human observers may overlook. These cues partially compensate for missing tactile information. The examples suggest that its sensitivity to particular visual details may even exceed that of human observers. Designing “tactile tasks” therefore requires more than judging whether their visual cues are readily apparent to people. As several tasks approach perfect scores, future benchmarks should control visual cues that can compensate for missing tactile information and introduce visually unobservable physical properties, tighter manipulation tolerances, and more complex contact and recovery sequences to distinguish models’ tactile understanding and manipulation capabilities.
The comparisons below show three classification and selection tasks with 100% success under both vision-only and visuo-tactile inputs. Key visual cues in the action notes are highlighted in green and key tactile cues in red, showing the evidence Astra uses to make judgments under each input condition.
Grasp Classify
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
Roughness Classify
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
Roughness-Guided Regrasp
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
Analysis of tasks whose success rates decreased with tactile input reveals misinterpretation of weak load cues and repeated probing that consumed the action budget, alongside possible stochastic variation. In the three paired Weight Classify cases where vision alone succeeded and visuo-tactile input failed, Astra interpreted small tactile shear or contact patterns resembling the light-object reference as evidence of a light load, misclassifying heavy objects. The vision-only runs instead reached the correct judgment from visible slip after relaxing the grasp. In Empty-Can Selection, subtle tactile changes likewise led to incorrect weight judgments, while repeated comparisons consumed the action budget and left some transfers incomplete. These cases identify the interpretation of weak tactile cues and the efficiency of probing as important factors in performance.
The paired examples below show vision-only successes and visuo-tactile failures in Weight Classify and Empty-Can Selection. Green highlights key visual cues such as visible slip; red highlights the tactile cues used by the model to judge weight.
Weight Classify
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
Empty-Can Selection
ACTION NOTE
COMMAND
ACTION NOTE
COMMAND
3.4 Astra still has limitations
The preceding examples demonstrate Astra’s perceptual and manipulation capabilities, but these capabilities remain inconsistent under complex contact conditions. Four limitations emerge: limited sensitivity to tangential motion and forces can lead to misinterpretation of subtle tactile cues; fine spatial relationships can be estimated inaccurately; plausible corrective intentions may not translate into effective actions; and repeated probing does not necessarily produce reliable convergence.
The following three visuo-tactile failures illustrate difficulties in physical-property inference, fine alignment and correction, and active probing. Together with the successful examples, they show that the challenge extends beyond proposing a plausible strategy to estimating contact state accurately and translating that judgment into reliable actions.
Limited sensitivity to tangential motion and force
Weight Classify
ACTION NOTE
COMMAND
Fine adjustments remain challenging
Insert Block v1
ACTION NOTE
COMMAND
Repeated probing without timely convergence
Empty-Can Selection
ACTION NOTE
COMMAND
Inference overhead and response latency
The visuo-tactile evaluation used approximately 59% more tokens and incurred 42% higher costs than the vision-only evaluation. Cached tokens accounted for over 80% of input tokens in both groups, allowing most inputs to be reused through caching. Together, the two evaluations consumed approximately 336 million tokens at a cost of CNY 7,505.
| Input modality | Input tokens | Cached input | Output tokens | Total tokens | Cost (CNY) |
|---|---|---|---|---|---|
| Astra (V) | 127,250,755 | 104,837,109 | 2,283,555 | 129,534,310 | 3,102.251 |
| Astra (V+T) | 203,911,092 | 170,693,217 | 2,521,218 | 206,432,310 | 4,402.802 |
| Total | 331,161,847 | 275,530,326 | 4,804,773 | 335,966,620 | 7,505.053 |
The two response-latency distributions are similar and both have long tails. Median latency is 15.4 seconds for vision-only input and 14.0 seconds for visuo-tactile input; 90% of valid requests complete within 46.3 and 44.0 seconds, respectively. Latency is the complete API response time recorded in provider logs, including server and gateway waiting time but excluding simulation execution.
| Input modality | Valid requests | Mean (s) | Median (s) | P90 (s) |
|---|---|---|---|---|
| Astra (V) | 3,927 | 22.8 | 15.4 | 46.3 |
| Astra (V+T) | 4,890 | 21.1 | 14.0 | 44.0 |
Toward general-purpose contact manipulation
Contact-rich manipulation and tactile perception are not inherent moats for specialist policies. With only a few demonstrations, GPT-6 Astra directly uses vision-based tactile images to outperform the reported specialist baselines in our comparison across two benchmarks, while exhibiting active probing, fine adjustment and manipulation recovery. Its vision-only performance also reveals subtle cues that human observers may overlook, demonstrating how vision can partially compensate for missing tactile information. Yet interpreting tangential motion and forces, estimating spatial relationships and correcting motion under contact constraints remain inconsistent. Future work should incorporate vision-based tactile sensing into joint pretraining with visual inputs and explore how these capabilities scale alongside foundation models. Tasks involving physical properties that cannot be distinguished visually, tighter manipulation tolerances and real-robot experiments will help establish whether these capabilities translate into reliable physical interaction.
Demonstrations, decisions, outcomes
Explore demonstration and evaluation videos by task, input modality, and random seed, with action explanations and commands that trace Astra’s manipulation process.
References
Citation
@article{yang2026tactalent,
title={GPT-6 Astra Is a Native Visuo-Tactile Policy for Contact-Rich Manipulation},
author={Yang, Chen and Shi, Linzhe and Zhang, Hang and Xu, Mu and Fan, Jiansheng and Wang, Chen},
journal={arXiv preprint arXiv:2610.26846},
year={2026}
}