Apr 2026 – Jun 2026
Real Fit Recommendation System
Start from one full-body photo, your height, and your weight. RFRS estimates your body, recommends a size, and shows a try-on image that reflects how it fits. Made by a six-person team at the 8th YAICON.
- My role
- Built the pipeline and crawled Musinsa data on a six-person team: 6,858 reviews and 9,297 images
- Stack
- Python, CatVTON, FLUX, SHAPY, Web Crawling
The problem
Size-related returns are a common problem in online clothing, and a size chart alone does not show how something will fit. Our goal was a try-on model built on real body measurements, with two contributions: measuring actual sizes from a photo alone, and reflecting the real sense of fit in the try-on image.
How it works
- Body measurement: SHAPY and SMPL-X rebuild a 3D body from a single RGB image, then chest, waist, hips, and height are calculated from the mesh.
- Fit classification: a fit classifier combines the body, the product size chart, and review text to label the fit as slim, regular, or loose.
- Virtual try-on: the fit label conditions CatVTON through a Fit-Aware Layout Generator and a Multi-Scale Fit Injector, and a FLUX denoising loop generates the image.
Data
We crawled Musinsa for top garments and buyer reviews: 6,858 reviews and 9,297 images in total. The training set came to 7,621 pairs (992 slim, 4,429 regular, 2,200 loose), labeled from official Musinsa fit data rather than by hand-written rules.
Result
The demo starts with a photo, height, and weight. It estimates the body, recommends a size with a fit score, and generates a try-on image. The demo model was trained on 5,893 reviews and 120 garments with CatVTON and LoRA.
My part
I worked on building the pipeline and crawling the data.
Next steps
- Extend try-on and size recommendation from tops to bottoms.
- Fine-tune CatVTON LoRA on measurement conditions to improve fit accuracy.
- Fully integrate precise body measurement with SMPL-X (Multi-HMR).