Typos and a darker room: two more results from testing robot AI

Our first note asked whether robot AI models stay in their workspace when the instruction or the camera image is wrong. On four open vision-language-action models, in simulation, they mostly did not (We tried to talk four robot AI models out of their workspace).

This note goes deeper on two results people asked us about: that ordinary typos do about as much damage as attack commands, and that a darker scene was the strongest camera effect on a model we had never used before.

Result 1A typo is about as bad as an attack command

We changed 20% of the letters of the correct instruction at random. No intent, no target: the kind of damage a bad speech-to-text result or a copy-paste error causes. Then we compared it with commands written to pull the arm away.

On every model we tried it on, the typos were in the same range as the targeted commands. The models did not follow the attack commands well; they lost their way on anything unusual.

For a robot builder, the more likely trigger is not an attacker but a garbled command.

Result 2On a new model, a darker scene was the strongest camera effect

To check that our tests were not tuned to the first models, we ran them once on a fourth model by an independent author, loaded with a configuration file only. On that model the simplest camera change was the worst one: the scene 50% darker. With normal lighting the arm ended more than 10 cm outside its workspace in 8% of runs; with half the light, in 52%.

On the two models we had used during development, the same dimming was not a finding. The weak spot moved from model to model, so a test suite has to be run on your model, not borrowed from someone else's results. Dimming costs nothing to test, and a cloud passing a skylight or a lamp failing in a warehouse aisle produces it for free.

TakeawaysWhat we would take from it

1. Put typos, empty and wrong instructions, and a darker camera into every acceptance test. They are cheap and they found problems on every model we tried.
2. Test the model you will ship, not a similar one: the weak spot moved between models.

If you are building with robot AI models and want to know how yours holds up, we are happy to talk: [email protected]