This toy box does something incredible with AI-generated video

AI video generation may be impressive on a technical level, but typing out a prompt doesn’t exactly feel like creative work. Interaction designer Hun Han wondered how he could make that more of a collaborative experience and that led him to develop something pretty incredible: the Hush toy box.

Hush is a small, enclosed lightbox for photography. Users pose inanimate objects — action figures, clay models, plants, and whatever else they can think of — inside the box, then close the lid. After that, the magic happens: Hush snaps a photo of the scene inside the box and feeds it as a prompt to a video generation AI.

The result is often fantastic, as AI models are now at a point where they do a very good job of generating and rendering realistic video. And in this case, that realistic video incorporates the real-world items in the box. Imagine your LEGO minifigs battling a clay dragon that you sculpted. That is exactly the kind of video Hush can produce and you get to be part of the creative process by deciding what to put in the box and how to pose those things within the scene. You also get control over day versus night and the simulated weather in the scene.

Kling v2.5 Turbo does the heavy lifting of video generation and a PC connects to that via the Replicate AI. The physical controls, including the weather selection dial and the Hall sensor that detects lid closure, connect to the PC through an Arduino. That board also controls the LED strips that illuminate Hush’s interior. The PC snaps a photo of the scene through OpenCV and a webcam. Finally, the rendered video results display on a repurposed iPhone 6, which is visible through a peep hole on the top of the box. 

When it comes to whimsy and entertainment, this might just be the best use of AI that we’ve come across.