Steering Vision-Language-Action Models for Safe Robotic Execution

Inference-time activation steering for pretrained VLA models to influence robotic behavior without retraining.

Technologies
Vision-Language-Action ModelsActivation EngineeringPyTorchROS
Activation Engineering imageActivation architechture imagecasting image

Context

Vision-Language-Action models can map visual observations and language instructions directly to robot actions, but their black-box structure makes post-deployment safety control difficult. This project investigated whether mechanistic-interpretability methods such as Activation Addition and Conditional Activation Steering could modify robot behavior at inference time, shaping trajectories or selectively applying safety constraints without full model retraining.

What I contributed

I worked on the navigation and VLA infrastructure side of the project, including an attempted StreamVLN setup, InternNAV and Isaac Sim setup/evaluation, and fine-tuning setup for InternVLA-M1. I also helped debug environment and dependency issues, supported robotic-arm testing, and contributed to the broader evaluation pipeline connecting VLA deployment, simulation, and activation-steering experiments.