Window Mode

Stack: TypeScript, WebGPU

A monitor is a flat rectangle showing a fixed image. Window Mode makes one behave like a hole cut in the wall. A webcam tracks where your head is and the scene is redrawn from where your eyes actually are, so near objects shift against far ones the way they do through a real window. That parallax is the whole illusion. The 3D content stops looking like a picture of a room and starts looking like a room, with no headset, glasses, or special screen. The demo we posted went viral on X, accumulating over 54k likes and 3.9 million views.

Filmed off-screen, since the parallax only reads from outside the monitor.

Highlights

Head tracking from an ordinary webcam, with no headset or depth sensor.
An off-axis projection pinned to the screen's corners, so moving your head looks through the scene rather than around it.
Depth from a single camera, estimated from the apparent size of the iris.
Face tracking kept off the render thread, so the image keeps up with your head.