VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation

Abstract

Multi-task manipulation requires picking the right action mode for the task at hand — the wrong pick means the wrong task. VQActFlow tokenizes action chunks into a discrete codebook and generates code sequences with Variational Flow Matching, keeping an explicit, steerable preference over action modes throughout generation. Classifier-free guidance and a learned codebook critic steer that preference at inference time — no retraining needed.

81.0% LIBERO-Goal peak success (w=4), vs. 61.5% for the closest discrete baseline

80.5% LIBERO-90 success with CFG + critic, best among all baselines

57.5% G1 humanoid success at w=6, up from 23.8% unguided

77.5% Bimanual avg. success with CFG + critic combined

Method

Tokenize, generate, steer

VQActFlow framework: VQ-VAE action tokenization followed by a Variational Flow Matching policy with CFG and codebook critic guidance. — **Figure 1.** A VQ-VAE tokenizes action chunks into a discrete codebook. A Variational Flow Matching policy generates code sequences while keeping logits over the full codebook at every step — a categorical preference that CFG and a codebook critic can steer.

Most flow-matching policies generate in a continuous space with no explicit notion of action mode, so guidance just reshapes one continuous distribution. VQActFlow keeps a discrete, steerable preference over action modes throughout generation, not just at the end of sampling.

1
CFG on language conditioning. Extrapolates conditional vs. unconditional logits at every step to sharpen commitment to the instructed task.
2
A learned codebook critic. A small contrastively-trained transformer scores feasibility and nudges the distribution toward feasible modes.

Both act on the same categorical preference, so their effects accumulate and combine without retraining.

Experiments

LIBERO, a G1 humanoid, and an ALOHA-style bimanual platform

VQActFlow outperforms continuous and discrete baselines across all three platforms, under matched encoders, data, and training steps.

CFG on LIBERO-Goal

Guidance through the categorical interface beats guidance through a continuous proxy.

LIBERO-Goal success rate vs. CFG weight for VQActFlow and Discrete Policy. — **Figure 2.** Success rate vs. CFG weight. VQActFlow peaks at w=4 (81.0%); Discrete Policy needs stronger guidance to reach a lower ceiling (61.5%).

Multi-task scaling on LIBERO-90

CFG and the codebook critic provide complementary, largely non-redundant gains at scale.

**Table I.** LIBERO-90 success rate (%) for different policies, trained from scratch under matched conditions.
Method	Type	CFG w	Critic λ	Success
MT-ACT	Continuous	–	–	72.4
CFM	Continuous	–	–	77.2
VQ-BeT	VQ-based	–	–	24.1
Discrete Policy	VQ-based	1.0	–	49.4
Discrete Policy	VQ-based	2.0	–	60.3
VQActFlow	VQ-based	1.0	0.0	72.3
VQActFlow	VQ-based	1.0	1.0	74.3
VQActFlow	VQ-based	2.0	0.0	77.6
VQActFlow	VQ-based	2.0	1.0	80.5

**Table II.** Ablation on codebook size K, LIBERO-90, guidance disabled.
Codebook size	128	256	512	1024
Success rate	62.6	65.6	72.3	70.1

Success peaks at K=512: smaller codebooks limit vocabulary granularity, larger ones enlarge the per-position classification space and make the categorical prediction harder to learn.

Humanoid whole-body manipulation

A Unitree G1 with two dexterous hands, four shared-workspace pick-and-place tasks differing only in target object and instruction.

Unitree G1 humanoid experimental setup and four pick-and-place evaluation tasks. — **Figure 3.** G1 humanoid setup and tasks: place the red ball / cup / bottle / medicine bottle into the box.

**Table III.** Aggregated G1 hardware policy success rate (%).
CFG weight	Success	Missed grasp	Wrong task
w=1	23.8	35.0	41.3
w=6	57.5	40.0	2.5

CFG's gain comes almost entirely from correcting wrong-task errors (41.3%→2.5%) — it improves task disambiguation, not execution quality.

Bimanual manipulation

Two ALOHA-style arms on four contact-rich tasks, three requiring coordinated bimanual use.

**Table IV.** Success rates (%) on bimanual manipulation tasks.
Method	Battery	Duck	Cylinder	Ball	Avg.
Discrete Policy, w=1.0	35.0	70.0	15.0	75.0	48.8
Discrete Policy, w=4.0	65.0	70.0	35.0	75.0	61.3
VQActFlow, w=1.0, λ=0.0	40.0	70.0	15.0	55.0	45.0
VQActFlow, w=1.0, λ=1.0	50.0	75.0	50.0	65.0	60.0
VQActFlow, w=4.0, λ=0.0	70.0	85.0	55.0	85.0	73.8
VQActFlow, w=4.0, λ=1.0	70.0	90.0	65.0	85.0	77.5

A continuous flow-matching baseline at the same backbone fails all four tasks — severe joint-jerk oscillation, roughly two orders of magnitude worse than VQActFlow.

**Table V.** Bimanual inference time per action chunk (ms), RTX 5090.
Configuration	Unguided	CFG	Critic	CFG + critic
Inference time	155.2	288.9	216.7	350.2

350 ms fits well within the 1.28 s per action chunk; asynchronous inference computes the next chunk during execution.

Citation

BibTeX

@article{zhao2026vqactflow,
  title={VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation},
  author={Zhigen Zhao, Mark Leggiero, Yipu Chen, Haoran Liu, Yifan Wu, Huishu Xue, Sirui Zhan, Ye Zhao},
  journal={arXiv preprint arXiv:2606.21600},
  year={2026}
}