The Existing App

Capture quality matters. The app already knows this.

The Output

Impressive! This works for a lot of homes.

What About Height?

But we don't live in a flat world ...

Why Height Matters

Two homes with identical floor plans can have wildly different energy needs.

The volume changes everything for HVAC sizing.

The Challenge

How do we take this 2D representation
and expand it into 3D space?

The Gold Standard

LiDAR

Light Detection and Ranging

Emits infrared pulses and measures how long they take to return. Direct distance measurement.

Apple's RoomPlan API + LiDAR would solve this easily.

The Accessibility Problem

LiDAR is iPhone Pro models only.

Validation

"Focusing on non-LiDAR is correct — that's a foundational design decision for us."

"Better spatial data for sizing would be the priority. The user is an Advisor."

— Justin Holmes

The Modeling Stage

Build vs Buy

Classic question. I explored both tracks in parallel.

The CUDA Wall

The Critical Distinction

After SfM, two paths diverge:

This maps to the dual audiences:

Suppliers/Advisors need geometry.

Consumers want beautiful visuals.

What Are Gaussian Splats?

A scene represented as millions of fuzzy ellipsoids.

Each ellipsoid has position, size, rotation, color, and opacity. Rendered by "splatting" them onto the screen.

Why they're popular:

• Photorealistic renders from any angle
• Fast to train (minutes, not hours)
• Real-time rendering possible

112K ellipsoids = photorealistic room

The Only Path Forward

With CUDA tools off the table, the only open source path was gaussian splats.

• COLMAP gave me sparse reconstruction
• Built OpenSplat with Metal support on Mac

The renders were ... compelling.

OpenSplat Experiments

n=2000, d=1, ssim=0.2

112K gaussians

n=2000, d=2, ssim=0.1

90K gaussians

n=3000, d=1, ssim=0.2

368K gaussians

n=5000, d=1, ssim=0.2

790K gaussians

Key insight: more iterations ≠ better quality.

Extraction

Tried to extract geometry via Poisson surface reconstruction.

Principal Component Analysis (PCA) normals

Estimate normals from neighboring points

Quaternion orientation normals

Interpret ellipsoid rotation as surface direction

Both attempts: garbage. Why?

Commercial Track: Back to Densify

Turned to commercial tools for reliable dense reconstruction.

KIRI Engine

Good gaussian splats, but inconsistent geometry. Struggled to reliably extract surfaces.

$500 minimum top-up

Polycam

Decent splats, but excels at reliable dense point clouds. Full photogrammetry pipeline.

$400/year Business

Polycam: 1.1M surface-bound points we can actually use.

Dense Point Cloud

1,120,180 surface-bound points

Polycam's photogrammetry gives us real geometry.

Points that actually lie on surfaces. Not optimized for appearance.

Now we can extract structure.

Poisson Mesh Extraction

24,174 triangles

With real surface points, Poisson reconstruction works.

Captures every bump, curve, and imperfection in the room.

Not that useful for visualization.

Probably overkill for Manual J.

What is it we actually need?

The Razor

Floor. Ceiling. Four walls.

That's it. That's all we need for volume calculation.

Plane Detection

Manhattan World: Walls perpendicular, floors horizontal, aligned to axes.

Floor:   lowest 2% of Z    |
Ceiling: highest 2% of Z   |  → 6 planes → 24 vertices
Walls:   X/Y boundaries    |

Simpler. More complete. No iterative fitting.

The Output

6 planes

{
  "room": {
    "width_ft": 23.5,
    "depth_ft": 24.6,
    "height_ft": 22.1,
    "area_sqft": 578.0
  },
  "calibration": {
    "method": "reference_height",
    "reference_height": 25.0,
    "scale_factor": 1.0
  }
}

Circle Back: The 25' Ceilings

Vaulted ceilings slope. Percentile bounds find the average, not the peak.

Is it perfect? No. Is it close? Pretty damn close.

For Manual J with 15-25% built-in tolerance? Good enough.

The Punchline

The simplest path that actually works.

What This Means for Zero Homes

The Next Challenge

Semantic segmentation to identify windows, doors, features.

SAM 2 is one promising tool. Early experiments show it can track surfaces across video.

SAM 2: Early Experiments

Walls tracked across video. Windows visible (over-segmented into panes).

Needs tuning, but the foundation is there.

Recommended Next Steps

Add complexity only when the problem demands it.