Synopsis

Autonomous vehicles are hard. The hardest part is creating an AI that can react naturally to the environment. Starsky Robotics was working on an autonomous solution with a clever twist. They would use a remote-driver for anything the AI couldn't handle.

I was responsible for making the teleoperation program succeed. I started with a rough prototype that could drive 10 mph on a straight road. When I finished, we had multiple control stations that could drive trucks in live traffic.

The challenge

The founders wanted to demo an MVP in less than 100 days

The demo would be a simulation of a gate-to-gate trucking run. We could only use teleoperation and the autonomous system. The safety driver could not intervene.

My role

INITIAL ASSESSMENT

The prototype required a team-effort to drive 10 mph on a straight road

UX & Product Design

Ergonomics

The easiest way to drive adoption for an experimental system is to make it feel familiar and natural

I relocated the control station to a private room. I upgraded all the equipment to make the driving experience more true to life. Microphones and speakers were installed so the drivers could talk directly.

Poor video quality was an ongoing challenge that required clever solutions. We split the cell signal between services to reduce dropouts. We applied a matrix algorithm to warp the video fit the screens. I repositioned the camera in the truck to reduce angular velocity misperception. It looked like they were driving much faster than they actually were.

UX Research & Design

Safety = confidence

The drivers needed complete situational awareness at all times. Not only what the truck was doing, but who (or what) was in control of the truck at any given time.

The system was prone to problems like disconnections and signal loss. I immediately added error alerts and system status indicators. They were prominently displayed in the teleoperation control station and safety driver HUD.

We established verbal handoff protocols. Drivers were required to confirm "I have control / you have control" during handoffs. They practiced safety drills at the start of every training session.

UX Research

Driver training

New drivers started by playing a video game (American Truck Simulator). This enabled them to become comfortable with the control station.

The hands-on training was experimental. We started with baby-steps: slow figure-eights, handoffs, and safety drills. Next, we started to address “stuff you can’t feel because you’re not in the truck” like controlled braking and turns.

Everything from there was a matter of progression. We moved from a closed lot to low-traffic areas and driving with a trailer. After that was driving on the highway, on-ramps, off-ramps, and eventually in street traffic.

Product Design

Designing for scale

I scaled the system to multiple control stations & trucks

The initial prototype was a one-to-one system with one truck linked to the station. An engineer had to update the code to switch to another truck.

I decoupled the teleoperation system into separate master and control stations. The master station could track and assign trucks. Tracking included location, speed, fuel levels, and cell strength for all trucks. The control station focused on driving systems like video and indicators.

A month before the demo, I opened a new teleoperation center, this time in Florida. The second location required new safety checks for conflict prevention between master stations.

Delivery

A successful demo

On September 22nd, 2017 Starsky Robotics completed the world's first end-to-end autonomous trucking run

Twitter • LinkedIn
Twitch • Mixcloud