Inter-X: Towards Versatile Human-Human Interaction Analysis

*Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, **Shuwen Wu**, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng, Yunhui Liu, Wenjun Zeng, Xiaokang Yang. IEEE Conference on Computer Vision and Pattern Recognition (**CVPR**), 2024.* [[paper]](https://arxiv.org/pdf/2312.16051.pdf) [[project]](https://liangxuy.github.io/inter-x/)

Inter-X is a large-scale human-human interaction MoCap dataset with ~11K interaction sequences and more than 8.1M frames. The fine-grained textual descriptions, semantic action categories, interaction order, and relationship and personality annotations allow for 4 categories of downstream tasks.

HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects

*Xintao Lv, Liang Xu, Yichao Yan, Xin Jin, Congsheng Xu, **Shuwen Wu**, Yizhou Zhou, Yifan Liu, Lincheng Li, Mengxiao Bi, Wenjun Zeng, Xiaokang Yang. European Conference on Computer Vision (**ECCV**), 2024.* [[paper]](https://arxiv.org/abs/2407.12371) [[project]](https://lvxintao.github.io/himo/)

HIMO is a large-scale MoCap dataset of full-body humans interacting with multiple objects, benchmarking two novel tasks of HOI synthesis conditioned on either the whole text prompt or the segmented text prompts as fine-grained timeline control.

SVD-InST: Image Style Transfer

GitHub｜ PDF

In this project, we build our SVD-InST model for image style transfer on Stable Diffusion 1.4 model. By introducing the SVDiff and text inversion finetuning technique, we achieve decent result on the given dataset.

Text Aligned SalGAN

GitHub｜ PDF

In this project, I tried to invent an arch combining CLIP and GAN to predict visual saliency. The model is trained on a dataset of eye tracking data where subjects are provided with text cues during the experiments.

Recommendation System & LLM Instruction Tuning

GitHub

Developed a recommendation system integrating various graph neural network algorithms (GraphConv, GCN, GAT, FT-Transformer, TabNet). Combined GraphSage and SkipGNN to improve existing algorithms. Conducted instruction tuning on GPT-J and GPT-3 models for movie recommendation tasks.

Spoken Natural Language Understanding

Conducted two complementary studies on Spoken Language Understanding (SLU): (1) a traditional deep learning-based semantic slot filling system and (2) a generative LLM-based information extraction framework. Implemented a BERT-RNN model for Chinese semantic slot filling, integrating data augmentation and noise filtering. Designed and tested multiple prompt engineering strategies for DeepSeek Chat, successfully enabling zero-shot structured extraction of semantic triplets.

3D Human Body Motion Estimation

🏆 Third Place on CVPR 2024 Ego-EXO4D Challenge

Actively participated in the Ego-EXO4D Challenge at the CVPR 2024 workshop, where our team secured third place among a competitive field of international participants. Developed a state-of-the-art 3D body motion estimation model leveraging the Coarse-to-Fine Vision Transformer (CF-ViT).

Empty Star - SandBox RPG

A SandBox RPG developed with Unreal Engine, where players take on the role of an explorer from an underground civilization, trying to traverse the Earth's crust and reach the surface.

AI Institute, School of Computer Science, Shanghai Jiao Tong University

Undergraduate Research Intern | July 2023 - May 2024

Supervisor: Prof. Xiaokang Yang, Prof. Yichao Yan

Research Focus: Computer Vision, Embodied Intelligence

• Data Collection & Processing: Managed data collection and participant coordination, developing visualized data processing tools for human action sequences with features for data slicing and text annotation. Resolved technical inconsistencies between hand-motion-capture gloves and Body MoCap system through manual offset correction.
• Model Training & Validation: Validated dataset reliability by training and testing advanced models (motion diffusion models, motion latent diffusion, text2motion). Managed multiple Docker containers for baseline models with tailored data preprocessing.
• Virtual Scene Development: Created and rendered multiple virtual scenes in Unreal Engine. Independently designed a virtual rendering pipeline using Unreal C++/Blueprint and Blender for batch-rendered animations.
• Research Contribution: Participated in and published two conference papers presented at CVPR 2024 and ECCV 2024.

Digital Intelligence Institute, China Pacific Insurance Co

Algorithm Intern | Feb 2025 - July 2025

Supervisor: Hui Wang

Research Focus: Natural Language Processing, Software Engineering

• RAG Set-up: Helped deploy LLMs and set up a RAG system with OpenAI style API.
• Multi-agent Platform Development: Developed multi-agent platform with agents for DataQA, DocQA, and Troubleshoot helper.