 |
Real-Time Neural Cloth Deformation using a Compact Latent Space and a Latent Vector PredictorChanhaeng Lee, Maksym Perepichka, Saeed Ghorbani, Sudhir Mudur, Eric Paquette, and Tiberiu Popa |
We propose a method for real-time cloth deformation using neural networks. The computational overhead of most of the existing learning methods for cloth simulation often limits their use in interactive applications. Employing a two-stage training process, our method predicts garment deformations in real-time. In the first stage, a graph neural network extracts cloth vertex features which are compressed into a latent vector with a mesh convolution network. We then decode the latent vector to blend shape weights, which are fed to a trainable blend shape module. In the second stage, we freeze the latent extraction and train a temporal latent predictor network. The temporal latent predictor uses a subset of the inputs from the first stage, ensuring that inputs are restricted to those which are readily available in a typical game engine. Then, during inference, the latent predictor predicts the compacted latent which is processed by the decoder and blend shape networks from the first stage. The latent predictor is the crucial component to speed up our inference time by replacing the resource-intensive graph neural network from the first stage. Our experiments demonstrate that our method effectively balances computational efficiency and realistic cloth deformation, making it suitable for real-time use in applications such as games.
Cloth synthesis, Deep neural network, Real time
@inproceedings{Lee:2024:Cloth,
author = {Chanhaeng Lee and Maksym Perepichka and Saeed Ghorbani and Sudhir Mudur and Eric Paquette and Tiberiu Popa},
title = {Real-Time Neural Cloth Deformation using a Compact Latent Space and a Latent Vector Predictor},
booktitle={Proceedings of the European Conference on Computer Vision (ECCV) Workshop on Computer Vision For Videogames (CV2)},
pages={1-16},
year = {2024}
}
Preliminary version of the paper.
Supplemental material related to the implementation
Pre-print version of the video:
