3D wall decor: from a wall photo to a 3D model and a bill of materials

This was client work for a home-services company — a real-time 3D product-visualization tool for their carpenters and the people who plan how a room, and specifically a wall, should look after decoration. It’s a big system that several of us worked on, so I’ll describe the whole thing but be clear about which parts were mine.

What it does

Someone photographs a wall. On that photo they draw the measurements — wall height and width, the width of a door, the size of a cupboard or built-in almirah, depths, and so on. Drawing on a tablet is far easier than typing coordinates, so the UI lets them sketch a line and label it “3.5 ft”.

From there:

1. The image and measurements go to an AI step (Gemini + OpenAI, in four stages: dimensions → panels → assets → cove lighting) that interprets the structure.
2. That feeds Blender (running headless) which builds an actual 3D model of the wall.
3. The model is shown in the browser with Three.js, where the user picks from a large catalog — paint, colors, woodwork, lighting, sockets.
4. A rule engine then computes cost and materials, optimizes the offcuts to reduce waste and customer cost, and the final render shows how the wall will look, with per-material pricing.

The point of all this is to take work off the carpenters and on-site staff and let a plan be priced and previewed before anyone cuts material.

What I worked on

I joined an existing system, so my contributions were specific improvements rather than the whole thing:

• Material differentiation. Everything used to render in the same grey regardless of whether it was woodwork, paint, or another finish. I made the materials render in their real, distinct colors so the preview means something.
• Photorealism. I moved the output from an obvious demo look toward using real images and rendering them so the result resembles the actual wall, not a placeholder.
• Resize and cost bugs. Resizing materials was unreliable, and after a resize the rule engine sometimes failed to recompute the cost. I fixed both so the price stays correct as the plan changes.
• Modularizing the code. A few files had grown past ten thousand lines each. I broke them into smaller, well-named modules (a few thousand lines at most) while keeping every test passing — which also made the AI coding agents we use far more accurate on those files.

Stack

FastAPI backend, Blender 5 headless for model generation, a 22-rule joint engine that produces the bill of materials, a React + Three.js frontend, and Gemini/OpenAI for the image analysis. Uploads go through Cloudinary or GCP storage.

What I took away

The lesson here was about working inside a large, unfamiliar codebase and making it better without breaking it — measuring twice, keeping tests green, and leaving files in a state where the next person (or agent) can actually work in them. The differentiation and photorealism changes were the visible wins; the modularization was the one that made everything after it easier.