How Xision predicts whether a garment will actually fit you, before you buy it.
A single smartphone scan becomes a measurement-accurate 3D digital body model. Real garments are then run through a cloth-physics engine on that exact body, so instead of a size chart the shopper gets a simulated try-on: how the garment drapes, where it pulls, a fit score, a recommended size, and a pressure map per body zone.
Overview
Online fashion returns are dominated by fit. Size charts are guesses, and a 'medium' means something different in every store, so shoppers over-order, return most of it, and the brand and the landfill absorb the waste. Xision replaces the size chart with a simulation: extract real measurements from an ordinary phone scan, build a body model from them, and physically drape the actual garment on it. The output is not a recommendation pulled from a lookup table. It is derived from geometry produced by a cloth solver running on that specific body.
Body measurement extraction
- A FastAPI service that fuses MediaPipe pose landmarks and segmentation masks with a MiDaS monocular depth map running under ONNX Runtime
- Silhouette widths give one axis and depth gives the other; Ramanujan's ellipse approximation converts the pair into true circumferences for chest, waist, and hip rather than width-only estimates
- Multi-view fusion across three capture angles, with SMPL-X parametric body fitting for the full measurement set
Measurements to a simulation-ready avatar
The obvious approach, importing the raw scan mesh, breaks the cloth solver, because a scanned mesh has no clean skeleton or skinning. I made the pipeline measurement-driven rather than mesh-driven: morph a simulation-safe parametric base avatar to the person's real measurements and export that as the drape body. That meant reverse-engineering the engine's undocumented measurement-parameter table and building a mapping layer to bridge girth-versus-width semantics, unit normalization, and aliases across three upstream schemas, with deterministic builds cached per body-model version so one build is reused across every subsequent garment.
Drape and fit derivation
- A job runner that loads a garment asset, drapes it on the user's avatar, and extracts every output (rendered views, 3D mesh, and pressure maps) with typed failure classification, so a missing or bad asset fails loudly instead of silently returning a default mannequin
- The physics engine emits geometry, not answers: I wrote the pure, unit-tested model that turns measurements plus a garment size chart into a fit score, size recommendation, confidence percentage, and per-zone verdicts
- Category-aware zone weighting (tops judged on chest and waist, bottoms on waist, hip, and inseam) with a robust Geman–McClure loss so a single bad measurement cannot dominate the score
- A validation harness that regenerates the full output set across a matrix of garments and body types and asserts fit, size, and zone expectations per cell, so simulation quality is a test that fails rather than a vibe
Engineering judgment
Most of the pipeline was gated on a licensed, Windows-only simulation engine that was unavailable for stretches of the build. Rather than wait, I split every stage into engine-dependent and pure-derivable halves and built the pure halves (measurement mapping, fit scoring, asset resolution, output classification) against real data with real unit tests. When the engine came online, integration was a switch-flip rather than a start. I also led the research spike for generating simulation-ready garments directly from ordinary store listings: an LLM emits a canonical tech-pack spec, a parametric pattern program is driven from it, and the result is converted into a real garment file through the engine's API, with both load-bearing assumptions proven end-to-end on the live engine before any production code was written.