Spring 2026 · Team research prototype
VISION
CART.
A camera. A cart.
A learned way forward.
A small robot cart that drives in front of you and keeps its distance, using one camera and a policy we taught it by driving it ourselves. I built the hardware and the system that drives it. My teammates taught it to follow.

- My role
- Hardware, power & the Pi control system
- Team
- Tianxiang Liu · Tyler Potsiadlo · me
- Context
- Boston College CSCI 3387 · Spring 2026
- Result
- Holds a ~1.4 m gap · 5 Hz on a Pi 3B
Field demo
Back and forth.
One of us walks toward the cart and away from it. Nobody is driving: the cart is running our AWR-refined policy on its own. It backs up when you come close, moves forward when you drift away, and turns to keep you centered.
The cart is the small red robot on the table.
From a bare chassis to a policy
How it came together.

March 17. Two driven wheels and a front pivot. It wobbled in turns, so the final cart drives all four wheels.

My four-view drawing of the final hardware: camera on a LEGO TECHNIC mast, Pi 3B on the upper deck, WAGO power distribution, four servo-driven wheels.

My wiring diagram. The Pi gets its own 5 V supply, each servo pair gets its own ~6 V pack, and every ground is tied together.
March 31. Driving it from the dashboard I wrote, with the Pi streaming its camera back to the laptop.

About 15 cm long. Small on purpose: a tabletop prototype to prove the pipeline before building anything full-size.

I wrote the first version (connect, camera, drive, e-stop). Tianxiang turned it into the recording tool we collected all our training data with.

The marker on the shirt gives exact distance and heading from geometry. That became the label the cart learned from.
Four runs from the cart’s camera: body keypoints, marker lock, the observation it acts on, and the wheel command it chose.
The AWR policy turning in place to keep the person centered.

How the original and AWR-refined policies respond to heading and distance. AWR turns harder and reverses more gently.
Two numbers in, two numbers out
Take away everything else and the cart only needs to know two things: how far away you are, and which way you are. It only controls two things: how fast its left wheels turn and how fast its right wheels turn.
The whole project is about getting those four numbers right on a Raspberry Pi 3B: 1 GB of RAM, no GPU. So we used one camera and nothing else. No wheel encoders, no IMU, no LiDAR. Distance and bearing come from geometry. Only the last step, choosing wheel speeds, is learned.
Why the Pi kept dying
The early versions ran the servos off the Pi’s own supply. A motor pulls the most current right when it starts, because it isn’t spinning yet and nothing is pushing back against the supply. Four servos starting at once dragged the voltage down far enough to reset the Pi. It happened every time the cart started, turned or reversed. It also ran hot, and it cost us more than one Pi.
A bigger battery wouldn’t fix that. The real problem was two very different loads sharing one supply. So I split them: the Pi gets its own 5 V supply, and each pair of servos gets its own ~6 V pack through WAGO connectors. All the grounds are tied together, so the Pi’s 3.3 V signal still reads as a valid pulse at the servos and no level shifter is needed. A heat sink and short sessions handled the heat.
After the split, the cart made it through every data-collection session and test run. I never measured rail voltage or servo current, though. That’s the first thing I’d add next time.
Stopping is harder than moving
Continuous-rotation servos treat a 1.5 ms pulse as “stop,” but every servo’s real neutral is a little off. So a stopped cart crept and jittered. You can chase the perfect neutral pulse for each servo forever. Instead, I stopped sending pulses at rest. No pulse, no motion. Commands smaller than 0.04 count as zero, and each side gets its own trim and direction setting.
Driving it from a laptop
I set the Pi up to run headless, with no screen or keyboard, so everything goes over Wi-Fi. A small web server on the Pi takes drive, stop and emergency-stop commands and streams the camera. A dashboard on the laptop connects to it by IP, so a teammate only needs a browser to drive the cart and see what it sees. One script copies the code to the Pi and sets it up over SSH.
The camera process crashed during early testing and froze the whole stream. So the agent now falls back through three ways of reading the camera and keeps retrying the real one every five seconds instead of hanging.
This is the platform the rest of the project ran on. The code is in my repository ↗.
Measuring a person with one camera
Tianxiang built the perception. For training, the person wears a printed 15 cm ArUco marker. Its four corners give the camera’s position relative to the marker, so distance is just the length of that vector and bearing is its angle. Calibration came out at 0.34 px of reprojection error.
One early calibration was done on the wrong lens and quietly stretched every distance by about 17%. Nothing crashed; the numbers were just wrong. Recalibrating on the real lens fixed it, and after that the camera mount stayed locked in place.
To drop the marker, a pose model finds the person’s shoulders and hips. Things look smaller the farther away they are, so distance goes as one over torso size. A two-parameter fit on that landed within 2.8 cm of the marker’s answer. A bigger machine-learning model didn’t do meaningfully better, so the simple fit shipped.
Teaching it by driving it
Writing down a reward for “follow comfortably” is hard. Driving the cart the way you want it to behave is easy. So we recorded ourselves driving it and trained it to copy us. That’s behavior cloning. The final dataset is six sessions, 449.5 seconds and 6,748 frames, with the marker locked 93.5% of the time.
The data needed more work than the model. About 73% of frames were “stop,” because whoever was driving mostly stood still while the cart settled. Capping each run of identical commands kept the model from just learning to stop. The cart is left-right symmetric, so every example was mirrored. That doubled the data and made sure it turns both ways equally.
At 5 Hz, every button tap lasts at least 200 ms, which spun the cart about 30° when you only meant a nudge. Dropping the spin command from 0.4 to 0.15 brought that down to about 10°. We also threw out a day of data from three different drivers, because their styles disagreed and the policy ended up fighting itself.
The policy itself is tiny: 4 inputs, two layers of 64, 2 outputs. It runs in plain NumPy on the Pi in about 50 µs per step.
Not trusting the easy number
A low training error only means the model copies the demonstrations well. It doesn’t mean the cart follows you well. Two policies with the same error can drive completely differently. So we scored the real thing: how far outside the follow band you got, how far off-center you were, and how jerky the wheel commands were. The band, 1.26–1.56 m, comes from how far we naturally stayed from the cart while driving it.
Tyler ran the policy on the cart and collected rollouts, and we refined it with advantage-weighted regression (AWR), which weights each step by how well it turned out. AWR cut jerk in half (RMS jerk down 33%, 0.079 → 0.053) but tracked distance about 21% worse, so the total score came out about even.
| Policy | Runs | Cost J | Distance term | Jerk term |
|---|---|---|---|---|
| Behavior cloning | 5 | 0.0238 | 0.0128 | 0.0006 |
| BC + AWR | 2 | 0.0264 | 0.0155 | 0.0003 |
Two AWR runs against five baseline runs is not much data, so I wouldn’t call either one better on that number. What the number misses is that the AWR cart was much calmer to walk in front of. Our next metric adds a term for that back-and-forth swooping.
What I’d do next
This is a 15 cm tabletop prototype for controlled indoor use. On the hardware side, I want rail voltage, servo current, Pi temperature, wheel speed and command-to-motion latency actually measured instead of assumed. Then a chest-height frame and a real drivetrain, so it can work as an actual cart. The learning side is ready for that already: drop the marker, flag lost-tracking frames in the input, and switch from regression to picking one of five actions.
Who did what
Me: the chassis and component integration, the power redesign, the headless Pi setup, and the original control system (Pi agent, servo driver, camera streaming, dashboard and deployment). Then hardware maintenance, camera replacements and manual driving for data collection through April and May, plus the hardware figures in our report.
Tianxiang Liu: perception and calibration, the markerless distance model, the data pipeline, the expanded dashboard, and the behavior-cloning and AWR training and evaluation.
Tyler Potsiadlo: our early reinforcement-learning frameworks, finding and selecting the working policy, and the on-cart rollouts for AWR.
Code: my hardware and control repository ↗ · team repository ↗. Boston College CSCI 3387, Spring 2026.
Explore more of the work.
← Back to Selected Builds