π0.5 on the edge at 43 Hz

Every action is a reaction

Real deployed robots demand a tight loop between observation and action. At even modest latencies, actions are executed against a state that has evolved, compromising both safety and robustness. Vision-Language-Action (VLA) models demonstrate impressive generalized manipulation capabilities but rely on a heavy LLM backbone and a costly iterative denoising process. These models must either reside on fast data center GPUs, incurring network latency / stability concerns, or slower edge devices. Prior efforts have thus sought to speed up VLAs by systematic transformer layer reduction [1], compressing multi-step diffusion into a single jump [2], as well as kernel tuning, layer fusion and quantization [3]. Notably, these approaches are complementary but have not been studied together. In this work, we synthesize approaches to massively speed up the powerful π0.5 [4] VLA. On the edge (an NVIDIA Jetson Thor) we're able to run π0.5 11× faster than the base model without regressing performance. This is 4× faster than NVIDIA's own attempt at optimized π0.5 inference [3].

We attack inference latency from several fronts.
Specifically, we combine Shallow-π [1] and SnapFlow distillation [2] with TensorRT compilation and quantization [3].

Throughout we evaluate on two simulated setups:

• LIBERO [5]: a widely used benchmark with 40 tasks covering spatial, object, goal, as well as longer horizon entangled objectives
• RoboLab [6]: a new photorealistic benchmark intended to address generalization and saturation in existing frameworks

Step 1: Shallow-π

First, we adopt the knowledge distillation technique Shallow-π [1]. π0.5 is composed of a 2.6 billion parameter 18-layer VLM prefix and a 300 million parameter 18-layer diffusion action expert, wherein each layer of the action expert attends to the corresponding layer in the VLM. Shallow-π uniformly subsamples the layers of both and then trains the subsampled (student) model using a combination of three losses:

• Standard flow matching loss: predict ground-truth velocity
  ℒfm = ‖ vˢ − vᵍᵗ ‖²
• Knowledge distillation loss: predict teacher velocity
  ℒkd = ‖ vˢ − vᵀ ‖²
• Attention distillation loss: KL divergence on the softmax attention scores between the student and teacher model for a specific layer
  ℒattn = KL( Aᵀ ‖ Aˢ )

In our work, we opt to forgo the attention distillation term. The paper's ablations demonstrate only a slight improvement and it was not used in the released code.

We explored distilling both 9-layer and 6-layer student models. In line with the paper's results, quality regressed slightly when distilling to just 6 layers. In the interest of matching baseline performance we proceed with a 9-layer model. We note however that a 6-layer model may offer an ideal latency/model-capacity tradeoff particularly when forgoing generalized capabilities and optimizing for specific tasks.

On data for distillation

Notably there is an associated LIBERO dataset consisting of simulated teleoperated trajectories that we can use for distillation. No such RoboLab dataset exists. Instead we use the real-world DROID [7] dataset for distillation on RoboLab tasks. The DROID dataset and RoboLab consist of the same setup (7+1 DOF Franka Panda with wrist + external cameras) but differ considerably in object vocabulary and task definitions. DROID is a massive in-the-wild dataset. Its demonstrations were assembled from dozens of collectors across 3 continents. It's a lot noisier than the LIBERO trajectories.

In the plots below we collect 4 random samples from the LIBERO dataset and 4 random samples from DROID. The solid lines show ground truth predictions for each joint. The dashed lines capture the corresponding teacher predictions. These are the standard flow matching and knowledge distillation regression targets, respectively. On LIBERO the predictions are similar, but the targets diverge quite a bit on DROID, particularly on samples where the robot is stationary. To stabilize training on DROID we use only the knowledge distillation loss and separately opt for a quality-filtered subset of DROID [8].

Shallow-π distillation halves inference latency on the Jetson Thor, from 260ms down to 127ms (p95).

Step 2: SnapFlow

Although the prefix VLM in a VLA is typically much larger than the action expert, the VLM runs only once per inference while the action expert runs multiple denoising passes (10 ODE steps in π0.5). Thus the smaller module dominates inference latency.
SnapFlow [2] is an intriguing new method for compressing the multi-step denoising into a single pass.

Folding the ten denoising steps into one, SnapFlow drops latency another 2.3×, down to 56ms.

Step 3: TensorRT

Shallow-π and SnapFlow reduce model size and denoising steps. But the resulting VLA is still large for an edge device like Jetson Thor, where memory bandwidth and available TFLOPs remain real constraints.

We use TensorRT [9] to compile the expensive parts of the policy into optimized CUDA engines. Given the model graph, TensorRT benchmarks candidate kernels on the target GPU and auto-selects the fastest for each operation, fuses adjacent layers to cut memory traffic and kernel-launch overhead, and picks tensor layouts tuned to the hardware.

We focus the TensorRT boundary on the two modules that dominate latency: encode-prefix and decode-denoise. Encode-prefix runs the VLM once to produce image/language embeddings and prepare the KV cache. Decode-denoise consumes that cache and predicts the action chunk.

Our first attempt compiled the graph with torch-tensorrt directly on Jetson Thor. It failed as Torch-TensorRT retained multiple copies of the policy and weights, and peak memory exceeded the edge device limit.

So we split compilation into two steps. First, we export the relevant PyTorch modules to ONNX. Then we pass the ONNX graphs to trtexec to generate TensorRT engine files. This lowers peak memory and makes the compilation pipeline easier to inspect.

The ONNX step also gives us explicit control over precision. We can verify the generated graph before TensorRT sees it, then ask trtexec to build strict-typed engines.

TensorRT compilation realizes a further 1.3× latency improvement, reaching 45ms.

Step 4: Quantization

After TensorRT, quantization is the remaining latency lever.

Jetson Thor supports BF16 and FP8 natively. We already run most of the model in BF16, while keeping numerically sensitive operations (softmaxes, norms) in FP32 to match the original PaliGemma/Gemma precision choices. The obvious target is the transformer MLPs, where FP8 can reduce compute and memory traffic without touching the most numerically sensitive operations.

We use NVIDIA ModelOpt between ONNX export and trtexec to produce FP8-calibrated graphs. Calibration data matters here. For the LIBERO fine-tuned model, we calibrate on LIBERO trajectories. For the RoboLab model, we likewise calibrate on DROID. Accuracy holds through quantization by preserving the activation ranges the deployed policy sees.

A final FP8 pass cuts latency roughly in half again, landing at 24ms.

Our final double-distilled+compiled+quantized model runs 11× faster than π0.5 on a modern edge GPU (NVIDIA Jetson Thor) while matching base model performance.

LIBERO

Across all 4 LIBERO task suites (goal, spatial, object, long-10) we maintain π0.5's strong performance.

RoboLab / DROID

On RoboLab, we likewise show the applicability of our approach in enabling fast edge inference while matching base model quality.
The more challenging RoboLab environments also demonstrate why low latency is critical.

We focus on two RoboLab tasks: BananaInBowl (simple) and Stack3RubiksCube (more challenging).

We first take the base π0.5 model and inject an artificial delay to model the inference latency in real-world deployment. When inference latency increases (from 66ms to 266ms) success rate on the BananaInBowl task drops slightly and measures of smoothness regress. We assess smoothness based on path length and SPARC [10]. SPARC (spectral arc length) is a frequency-domain smoothness metric that measures the arc length of the velocity profile's normalized frequency spectrum, where values near zero indicate smooth, efficient motion and more negative values indicate jerkier trajectories.

BananaInBowl — π0.5

It's not a drastic regression, but low latency becomes more critical as we move from simple to more dexterous tasks.

On the Stack3RubiksCube task, we observe a much starker latency-induced performance drop. On the π0.5 model, success rate drops from 44% to 35%.

Stack3RubiksCube — π0.5

Applying the composed optimizations, we matched performance on the BananaInBowl task. However, success rate dropped for Stack3RubiksCube on the DROID-distilled Shallow-π model. This is possibly because the Shallow-π model has fundamentally lower capacity. However, it could also be that our dataset didn't adequately cover Stack3RubiksCube-like abilities and/or our training budget was exhausted before the model could recover this behavior.

We're able to recover (in fact, exceed) teacher performance with a simple expert behavior-cloning (BC) recipe. Specifically, after distillation on the DROID dataset we additionally train briefly on a mix of DROID and successful Rubik's stacking trajectories collected from teacher model rollouts. This expert-BC distillation can be done quickly on a small dataset. We collect fewer than 100 rollouts and train for just ~5% of the steps of the initial DROID distillation.

Stack3RubiksCube — Optimization Stack

The SnapFlow paper exclusively trains and evaluates on LIBERO. So we are excited to demonstrate SnapFlow successfully adapted to the more challenging RoboLab environments.

Guild is working on making robots deployable in the real world.

Fast, on-board inference is critical for robust and safe manipulation.
Real-world navigation poses even stricter fault-tolerance and latency demands.

And we're not just pushing the frontier of optimization.
To realize our mission of robotic deployment at scale, we're building foundational navigation world models.
Excited to share more soon.