From raw frames to video, in one command.
The three outdoor scenes of the Mundo project page, section 1.1, each taken once through the deployment pipeline: the raw frames are reconstructed into a proxy scene, a camera path is designed from the scene, the proxy is rendered along it, and the video is generated. The number of chunks (one to three) and both seeds were drawn at random before the runs; every result on this page is the single run of its scene.
Stage by stage
Each scene shows what the stages wrote, in order: the input frames, the reconstruction as a point cloud with the boxes of the exported scene (drag to orbit, scroll to zoom, pick an object to highlight its box), the proxy video next to the generated video on the same camera path, and for every chunk the context photographs chosen and the description written for the prompt.
Input frames
Reconstruction: point cloud and boxes
Camera path: proxy and generated video
Context photographs and written prompt per chunk
The same historic abbey complex perched on a rocky island, surrounded by a tidal flat and dense green trees under a clear blue sky. the main abbey church with its tall spire and gothic architecture dominates, with stone buildings and ramparts forming a fortified village below. the light is warm and golden, casting long shadows. The whole path: a smooth camera move in which the camera slides left 1.6 m and sinks 0.8 m and turns right 15 degrees.
The camera slides left, sinks slightly, and turns right, revealing the abbey church's upper body and spire centered in the frame, with the ascending village curtain wall and lower wooded slope on the right, and the lower village curtain wall and rampart corner tower appearing on the left as the view progresses. Warm golden light casts long shadows across the stone buildings and tidal flat under a clear blue sky.
Input frames
Reconstruction: point cloud and boxes
Camera path: proxy and generated video
Context photographs and written prompt per chunk
The same ornate rotunda with classical columns and a domed roof, surrounded by water and lush greenery under bright blue skies, featuring pinkish stone and detailed carvings, with sunlight casting clear shadows. The whole path: a smooth camera move in which the camera slides right 4.6 m and rises 1.1 m and turns left 38 degrees, then pulls back 7.3 m and slides right 2.1 m and turns left 41 degrees, then pulls back 5.4 m and slides right 4.4 m.
The camera glides forward and slides right, passing close to the rotunda's northwestern pier, revealing its pinkish stone and detailed carvings, while the northeastern pier remains visible to the left in the mid-frame. The camera slides right and glides forward, turning left slightly, bringing the western pier into close view on the right while the northeastern pier recedes to the left background. The camera slides right and pulls back, turning left, revealing the western shoreline trees on the right and the southwestern pier in the mid-frame, with the upper drum and cornice visible in the distance. The camera pulls back and slides right, turning left, revealing the western shoreline trees in the center and the rotunda's northwestern and western piers in the mid-frame, with the northeastern pier visible in the far left background. Bright sunlight casts clear, sharp shadows across the pinkish stone of the rotunda and surrounding greenery under a vivid blue sky.
The camera pulls back and slides right, revealing the rotunda's western pier on the left, then the upper drum and cornice centered in the frame, while western shoreline trees appear on the right and front shoreline tree clumps are visible in the distance. The camera pulls back further and sinks slightly, turning left to show the far western tree and canopy on the left, with the rotunda's upper drum now small and far to the right, and the western shoreline trees remaining on the right. Bright sunlight casts clear shadows on the pinkish stone of the rotunda and the surrounding lush greenery under a clear blue sky.
The camera pulls back and sinks slightly while turning left, revealing more of the rotunda's western pier and the far western tree, which is large and close on the left, as well as the western shoreline trees and distant western column in the background. The camera pulls back slightly and slides right, bringing the front shoreline tree clump into view on the right while the rotunda's upper drum and cornice remain centered in the frame. The camera slides right and pulls back slightly while turning right, revealing the near bank tree on the right and the western shoreline trees on the left, with the rotunda's upper drum and cornice now on the left side of the frame. Bright sunlight casts clear shadows across the pinkish stone of the rotunda, the green foliage, and the calm water under a clear blue sky.
Input frames
Reconstruction: point cloud and boxes
Camera path: proxy and generated video
Context photographs and written prompt per chunk
The same brick lighthouse station with a tower, adjacent museum hall, and flagpole under clear blue sky. red brick, white trim, and dark green shutters define the architecture. wooden benches, a picnic table, and a bicycle are present. the american flag flies. bright sunlight casts sharp shadows. The whole path: a smooth camera move in which the camera slides right 3.1 m and sinks 1.3 m and turns left 36 degrees, then slides right 3.5 m and rises 1.1 m and turns left 38 degrees.
The camera slides right, sinks lower, and turns left, passing the brick museum hall on its left and the lighthouse tower on its right, revealing the flagpole and American flag in the center, then moving closer to the lighthouse tower which becomes large and central in the frame. Bright sunlight casts sharp shadows across the red brick buildings, wooden benches, and picnic table under a clear blue sky.
The camera slides right, rises slightly, and turns left, passing the lighthouse tower which dominates the center of the frame, revealing the brick museum hall to its right, and the flagpole with the American flag on the left. The camera passes the front landing brick base and ramp brick curb on the left, and the lighthouse lantern and gallery base becomes visible near the center as the camera turns. Bright sunlight casts sharp shadows under a clear blue sky.
Runs
One cold-start run per row on one eight-GPU node, all times in seconds. Wall clock covers the whole command. Model loading is the captioner and the condition encoder; prepare is the context selection, proxy rendering, prompt writing and condition encoding of all chunks; sampling is the diffusion steps and decoding of all chunks. What remains of the wall clock is loading the video model and the hand-over between stages.
| scene | chunks / frames | wall clock | reconstruction | path design | model loading | prepare | sampling |
|---|---|---|---|---|---|---|---|
| Mont-Saint-Michel | 1 / 124 | 981 | 645 | 55 | 73 | 28 | 123 |
| Palace of Fine Arts | 3 / 304 | 1349 | 700 | 65 | 68 | 64 | 395 |
| Brick lighthouse | 2 / 214 | 1099 | 604 | 40 | 76 | 48 | 268 |
Blocking issues are the objects the reconstruction's final review still rejects. Clearance is the smallest distance between the camera path and a proxy surface. Coverage is the share of each chunk's view that its context photographs show, one value per chunk. Seeds are the path seed and the sampling seed.
| scene | environment | objects | blocking | path m | clearance m | coverage | seeds |
|---|---|---|---|---|---|---|---|
| Mont-Saint-Michel | outdoor | 39 | 6 | 2.4 | 1.60 | 0.73 | 2398 / 43488 |
| Palace of Fine Arts | mixed | 54 | 13 | 18.7 | 0.32 | 0.51, 0.55, 0.38 | 3308 / 60471 |
| Brick lighthouse | outdoor | 31 | 7 | 6.5 | 0.68 | 0.54, 0.51 | 5488 / 95649 |
Settings
- One command per scene:
python -m mundo_infer.run all --source <frames> --auto --auto-seed <seed> --seed <seed> --chunks <n>, with the default settings of the package and no reconstruction prompt. - Reconstruction: 48 frames per scene, Pi3X geometry, one editing round; indoors or outdoors is decided by the pipeline itself.
- Generation: 640×480 at 24 fps, 124 frames for the first chunk and 90 more per further chunk, 32 steps of dpmpp2m, eight GPUs in tensor parallel, at most nine context photographs per chunk.