Article Details

Title · Authors · Abstract · Keywords · DOI · References

Virtual Try-on Based on Appearance Flow and Diffusion Model

CHEN Xiangyi

(The School of Information Science and Technology, Southwest Jiaotong University, Chengdu 610031, China)

Section：Artificial Intelligence · Vol. 2025, Issue 5 · Pages：17-24 · 2096-4706（2025）05-0017-08 · CLC number：TP391.4 · Document code：A · DOI：10.19850/j.cnki.2096-4706.2025.05.004

Abstract

This paper investigates the research status of virtual try-on field at home and abroad, including the application situation of technologies based on Generative Adversarial Network and Diffusion Model, and summarizes various datasets of virtual try-on. Based on related theories, it studies the virtual try-on based on appearance flow and Diffusion Model. Firstly, the Local-Flow Global-Parsing (LFGP) model is used to transform the target clothing to conform to the human posture. Then, the Diffusion Model is used to synthesize the characters and clothing, and the details are processed to complete the final try-on. Qualitative and quantitative analysis of the experimental results show that compared with previous studies, clothing transformation based on LFGP can better handle the clothing details and the connection parts between characters and clothing, and the Diffusion Model can also improve flexibility during the try-on synthesis processing while maintaining the authenticity, providing reference for further research of virtual try-on technologies.

Keywords

virtual try-on; Deep Learning; Diffusion Model; appearance flow

References

  1. [1] EHARA J，SAITO H. Texture Overlay for Virtual Clothing Based on PCA of Silhouettes [C]//2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.Santa Barbara：IEEE，2006：139-142.
  2. [2] TANAKA H，SAITO H. Texture Overlay onto Flexible Object with PCA of Silhouettes and K-means Method for Search into Database [C]//MVA 2009 IAPR Conference on Machine Vision Applications.Yokohama：MVA，2009：5-8.
  3. [3] HAN X T，WU Z X，WU Z，et al. VITON: An Image-based Virtual Try-on Network [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：7543-7552.
  4. [4] WANG B C，ZHENG H B，LIANG X D，et al. Toward Characteristic-Preserving Image-Based Virtual Try-on Network [C]//Computer Vision-ECCV 2018.Munich：Springer Nature，2018：607-623.
  5. [5] HAN X T，HUANG W L，HU X J，et al. ClothFlow: A Flow-based Model for Clothed Person Generation [C]//2019 IEEE/CVF International Conference on Computer Vision.Seoul：IEEE，2019：10470-10479.
  6. [6] CHOI S，PARK S，LEE M，et al. VITON-HD: High-resolution Virtual Try-on via Misalignment-aware Normalization [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville：IEEE，2021：14126-14135.
  7. [7] LEE S，GU G，PARK S，et al. High-resolution Virtual Try-on with Misalignment and Occlusion-Handled Conditions [C]//Computer Vision-ECCV 2022.Tel Aviv：Springer Nature，2022：204-219.
  8. [8] XIE Z Y，HUANG Z Y，DONG X，et al. GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-flow Global-parsing Learning [C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Vancouver：IEEE，2023：23550-23559.
  9. [9] HO J，JAIN A，ABBEEL P. Denoising Diffusion Probabilistic Models [J/OL].arXiv:2006.11239 [cs.LG].（2020-06-19）.https://arxiv.org/abs/2006.11239v2.
  10. [10] BALDRATI A，MORELLI D，CARTELLA G，et al. Multimodal Garment Designer: Human-centric Latent Diffusion Models for Fashion Image Editing [C]//2023 IEEE/CVF International Conference on Computer Vision.Paris：IEEE，2023：23336-23345.
  11. [11] GOU J H，SUN S Y，ZHANG J F，et al. Taming the Power of Diffusion Models for High-quality Virtual Try-on with Appearance Flow [C]//31st ACM International Conference on Multimedia.Ottawa：ACM，2023：7599-7607.
  12. [12] MORELLI D，FINCATO M，CORNIA M，et al. Dress Code: High-resolution Multi-category Virtual Try-on [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.New Orleans：IEEE，2022：2230-2234.
  13. [13] ZHOU T H，TULSIANI S，SUN W L，et al. View Synthesis by Appearance Flow [C]//Computer Vision-ECCV 2016.Amsterdam：Springer Nature，2016：286-301.
  14. [14] WANG M H，WANG Q，CHANUSSOT J，et al. L0-L1 Hybrid Total Variation Regularization and its Applications on Hyperspectral Image Mixed Noise Removal and Compressed Sensing [J].IEEE Transactions on Geoscience and Remote Sensing，2021，59（9）：7695-7710.
  15. [15] ROMBACH R，BLATTMANN A，LORENZ D，et al. High-resolution Image Synthesis with Latent Diffusion Models [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans：IEEE，2022：10674-10685.
  16. [16] CHENG B W，MISRA I，SCHWING A G，et al. Masked-Attention Mask Transformer for Universal Image Segmentation [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans：IEEE，2022：1280-1289.
  17. [17] GE Y Y，SONG Y B，ZHANG R M，et al. Parser-free Virtual Try-on via Distilling Appearance Flows [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville：IEEE，2021：8481-8489.

About the authors

Chen Xiangyi (April 2002-), female, Han ethnicity, from Shehong, Sichuan, is currently pursuing a master's degree in deep learning and image processing.