Abstract:
Synthesizing human motions is one of the holy grails of computer graphics and computer animation. The process of generating motions manually in the past was both laborious and time-consuming. The animation could be exaggerated and may not follow physics. To achieve more realistic and natural motions, people synthesize realistic human motions utilizing expensive and unwieldy MoCap equipment. The graphics community, in the last decades, has been committed to providing cheaper alternative solutions for generating realistic human motions. With the advent of large-scale datasets readily available to users, data-driven methods achieve magnificent success in generating motions at a much cheaper cost. Recurrent neural network-based (RNN-based) methods are extensively used to generate new motions by learning temporal relationships between adjacent poses. Initially proposed to tackle natural language processing (NLP) problems, transformers can be adapted to address the problem of motion generation and achieve state-of-the-art (SOTA) performance. In this report, we will demonstrate AI-driven knowledge-based motion synthesis algorithms and their novel contributions to the graphics and animation communities.
The major challenges in AI-driven knowledge-based algorithms are threefold. (1). In the SOTA, loss functions are usually based on empirical observations rather than theoretical deductions. It's still unclear how to justify the validity of the generated data. (2). The parametric models employed in the SOTA find it difficult to approximate non-linear real-life problems due to the limited number of parameters and sometimes the poor quality of the initial assumptions. (3). Most SOTA mainly focuses on intra-class problems. Generating poses that connect keyframes from different semantic activities is less discussed.
First, we presented a hybrid data-driven and model-driven method called dynamic motion transition (DMT). By augmenting the motion data with force terms, the principle of least action (PLA) is employed to justify the validity of the generated data, and dynamic movement primitives (DMP) are utilized to guarantee the convergence of the system. Second, we presented a non-parametric method motion Gaussian process (MGP). It predicts upcoming poses as multivariate Gaussian distributions. We devised a Gram matrix-based kernel to capture the underlying structure of the mapping. Human-in-the-loop strategy is used to adjust the results by moving the mean value within the confidence interval. Third, we present our novel transformer-based motion generator which is integrated with a so-called semantic motion ID system in the ongoing work. The ID system provides abundant information to help generate inter-class motions. Moreover, we also present a novel generative neural network, BodyGAN, to generate human motions in the form of detailed 3D human shapes given monocular RGB images only.
We conclude this report by discussing the major challenges in AI-driven knowledge-based motion synthesis, our plans to solve them, and future directions.
