VEDARSHMISHRA

November 2026 · Interactive installation

UNDER THE
SAME SKY.

A night sky on the floor
that connects the people on it.

Step onto the floor and a ring of light opens at your feet. Stop near someone for a second and a thread of light grows between you. Walk apart and it dissolves back into stars. Built in one afternoon at the Hatchery with Vlad Hoila.

See it on the floor ↓Code on GitHub ↗Project website ↗
The projected sky with five people connected by threads
UNDER THE SAME SKY / LIVE RENDER
My role
Software & tracking
Team
Vlad Hoila · me · Hatchery Group 4
Built
One afternoon · November 15, 2026
Built with
Python · YOLO11 pose · OpenCV · Pygame

From a taped square to a gathering place

How the afternoon went.

The finished floor
01 / 12
The finished floor

The installation in the Hatchery: a taped floor, two poster easels, and the sky projected between them.

11:12 · First projection
02 / 12
11:12 · First projection

The phone camera’s first look at the projection on the floor, streamed over Wi-Fi.

11:39 · First calibration
03 / 12
11:39 · First calibration

Clicking the four corners of the tape square so the software knows where the floor is.

15:02 · Someone steps on
04 / 12
15:02 · Someone steps on

The camera finds the feet (green) and the sky draws a ring under them.

15:30 · Two people
05 / 12
15:30 · Two people

Testing whether a thread forms between two people who stop near each other.

The ring missed
06 / 12
The ring missed

The camera sees the person, but the ring lands beside their feet instead of under them.

Markers, first try
07 / 12
Markers, first try

Projected ArUco markers for auto-calibration. The far half was too squashed for the camera to read.

Markers, pre-stretched
08 / 12
Markers, pre-stretched

Markers drawn taller at the far end so they look square from the camera. All four quadrants read.

Checking the fit
09 / 12
Checking the fit

Projector footprint (magenta), tape (blue), and four corner targets plus a centre spot, each checked against where it should land.

Under the feet
10 / 12
Under the feet

After recalibrating and teaching placement: the ring sits under the person.

A thread
11 / 12
A thread

Two people pause near each other and a line of light grows between their rings.

A constellation
12 / 12
A constellation

Five people, a few threads each, and a quiet question beside one of them.

It started with a question

We didn’t start with a projector. We started by asking people: when have you felt connected to something bigger than yourself, without anyone telling you what it meant? The answers were singing with family, looking up at the night sky, sitting in a temple, making something by hand. None of them came with instructions.

So the brief became: give people something to do together, and leave the meaning to them. A night sky belongs to no single tradition, and a floor puts people face to face instead of side by side at a screen. It only gets beautiful when people are near each other.

The floor only needs one point per person

For the sky to react, the software has to know one thing: where each person is standing. A pose model (YOLO11) finds 17 body keypoints per person, and the ankles are the ones closest to the floor. So each person’s position is the average of their confident ankle points. If neither ankle is confident, it falls back to the bottom-centre of their bounding box.

Pose detection runs on its own thread at whatever rate the laptop can manage, while the sky renders separately at 45–60 fps, so a slow frame of tracking never makes the floor stutter.

Camera pixels to floor metres

The camera and the projector look at the floor from completely different angles. But the floor is flat, and any two views of a flat surface are related by a single 3 × 3 transform, a homography. Four matching points pin it down. That one transform carries every foot from camera pixels to floor coordinates.

Then everything is in metres, not pixels: ring size, how far a ripple travels, how close two people need to be. A bigger floor still feels human-sized, because 1.6 m means 1.6 m whatever the projector does.

A thread has to mean something

If every pair of people within reach got connected, walking past someone would count as meeting them. So a thread only forms when two people are within 1.6 m, moving slower than 0.45 m/s, for about a second. It breaks at 2.1 m, not 1.6 m. The gap stops a thread from flickering on and off when two people stand right at the edge.

Tracking drops out sometimes. A lost person is held in place for 0.8 s, and someone who reappears within 3 s is treated as the same person: same colour, no new arrival wave. Orbits never spin faster when people connect; the whole thing is supposed to stay calm.

The ring kept missing the feet

The hardest part wasn’t the visuals. It was getting the ring to land exactly under someone. Clicking four corners by hand was slow and a little off every time; I recalibrated six times that afternoon. So I built auto-calibration: the projector shows numbered ArUco markers, the camera finds them, and the software fits the mapping and throws out bad reads.

Then the next problem showed up. The camera sat low, so far markers looked squashed and whole quadrants came back empty. Instead of moving the camera, I drew the markers pre-stretched, taller at the far end, so they look square from where the camera is. And the phone feed ran about 2 seconds behind, so the software was reading the previous pattern. Now it waits until the camera actually shows the new grid.

Rings were still slightly off, because ankles sit above the floor and the phone lens bends the image. So there’s a teach mode: someone stands still, you click where the ring should be at 5 to 9 spots, and the software corrects the whole floor. Calibration is verified by projecting four corner targets and a centre spot. If any one misses, nothing is saved and the old calibration stays.

Testing with nobody on the floor

You can’t keep asking friends to walk around while you change a glow value. So I wrote a scripted simulation of zero to seven people (empty, solo, pair, triangle, web, crossing, leaving) and automated tests that check every stage: no flicker, no stale threads, nothing drawn outside the taped area, and calibration left untouched.

Built in one afternoon

The first version was a small taped square, a projector pointed at the floor, and a phone as the camera. Room light washed out the sky, people’s own shadows cut across their rings, and the square was too small for more than two people. Each of those became a change: brightness controls tuned live from the laptop, a different projector position, a bigger floor.

TimeWhat happened
11:12First projection on a small taped square, seen through the phone camera
11:39First calibration
12:19First sky rendered by the code
15:02First person on the floor gets a ring
15:30Two people on the floor, testing the thread
16:07Posters up around the floor
16:09Live: the ring follows a foot

Who did what

Me: the software and tracking. The pose-tracking pipeline, rendering the sky, rings, threads and the three scenes, calibration, auto-calibration and teach placement, and the simulation and tests.

Vlad Hoila: concept, space and testing. The guiding question and the experience design, the posters and visual identity, the physical setup (projector, tape, easels, camera), and walking the floor to test and tune the tracking.

Code on GitHub ↗ · Project website ↗

Explore more of the work.

← Back to Selected Builds