I don't think that's true. AI could work around all the physical imperfections.

Just imagine yourself controlling an imperfect robot with some joysticks or maybe a sensor suit. You'd certainly do much better than current robots. The limit is the robot's brain, not the physical actuators/sensors.