Yue (Jack) Ma

Master Student

Tsinghua University
Beijing, China.

Email: mayuefighting [at] gmail.com

Biography

I am a year-3 master student at Tsinghua University, under the supervision of Prof. Xiu Li. I obtained my B.Eng in Computer Science at Taiyuan University of Technology in 2021. I studied in the MMLab@SIAT led by Prof. Dr. Yu Qiao. Currently, I am a research intern at HKUST, supervised by Qifeng Chen. I was also fortunate to be an internship at Baidu, Tencent AI Lab.

My research interests lie in the intersection of Computer Vision and Machine Learning. From 2021, I started to do some research on video understanding and self-supervised learning. Now, I focus on designing novel applications for image/video generation(Follow-Your-Pose, Follow-Your-Click), video editing(MagicStick). and other downstream AIGC tasks.

Feel free to contact me by email if you are interested in discussing or collaborating with me.

News

[03/2024] Excited to release Follow-Your-Click, the first framework to achieve regional image animation. Enjoy the fun of FollowYourClick !
[01/2024] One paper are accepted by CVPR 2024 !
[12/2023] Two paper are accepted by AAAI 2024 !
[11/2023] Excited to release MagicStick, the first unified framework to modify video properties leveraging the keyframe transformations on the extracted internal control signals.
[10/2023] One paper is accepted by TMM 2024 !
[09/2023] I am serving as a Reviewer for ICLR 2024 !
[07/2023] I start to cooperate with Qifeng Chen closely in HKUST !
[06/2023] I end my research intership in Tencent AI Lab !
[06/2023] I am serving as a Reviewer for NeurIPS 2023 !
[05/2023] I am serving as a Reviewer for ACM MM 2023 !
[04/2023] Excited to release Follow-Your-Pose framework, generating the character videos from pose and text description. Enjoy the fun of FollowYourPose !
[03/2023] Two paper are submitted to ICCV 2023 !
[02/2023] One paper SemanticAC about Audio Classification was accepted by ICASSP 2023 !
[12/2022] Awarded First-Class Scholarship of Tsinghua Shenzhen International Graduate School !
[11/2022] One paper about video-text pretraining with masked autoencoder is submitted to CVPR 2023 !
[10/2022] One paper about semantics-assisted framework for audio classification is submitted to ICASSP 2023 !
[09/2022] Attending ACM MM 2022, Welcome to chat!
[09/2022] My first paper has been accepted for Oral Presentation on ACM MM 2022 (Top 5%) !
[05/2022] Joined Tencent AI Lab as research intern !
[03/2022] Awarded Tencent Rhino-Bird Research Elite Program from Tencent ! Only 72 students in the world admitted to this program !
[07/2021] Joined CUHK@MMLab in ShenZhen as research intern !
[06/2021] Recommended to Tsinghua University towards the MSc degree !
[06/2021] Graduated from Taiyuan University of Technology !
[01/2021] Joined Baidu Research as research intern. Started doing research on Computer Vision !

Industrial Experience

Tencent Hunyuan

Dec. 2023 - Present, Tencent Hunyuan, ShenZhen, China

worked with Dr. Wei Liu

Topic: Video Generation (Follow-Your-XXX Family) and Video Editing

Tencent AI Lab

May. 2022 - July. 2023, Tencent AI Lab, Visual Center, ShenZhen, China

worked with Dr. Tianyu Yang, Dr. Xiaodong Cun and Dr. Xintao Wang

Topic: Video Generation and Video Editing

Baidu VIS

Jan. 2021 - July. 2021, Baidu, Visual group, ShenZhen, China

worked with Zhikang Zou

Topic: 3D Object Detection, 3D Image Understanding

Education & Visiting

	The Hong Kong University of Science and Technology, Hong Kong PhD Student in Visual Intelligence Lab, HKUST Advisor: Prof. Qifeng Chen Sep. 2024 - Future
	The Hong Kong University of Science and Technology, Hong Kong Research Assistant in Visual Intelligence Lab, HKUST Advisor: Prof. Qifeng Chen July. 2023 - Nov. 2023
	Tsinghua University, China Master of Engineering in Electronic Information Advisor: Prof.Xiu Li Sep. 2021 - Jun. 2024
	University of Chinese Academy of Sciences, China Research Assistant in Multimedia Laboratory, SIAT, CAS (MMLab@SIAT) Advisor: Prof. Yu Qiao and Dr. Yali Wang Jul. 2021 - Apr. 2022
	Taiyuan University of technology, China Bachelor of Engineering in Computer Science Sep. 2017 - Jun. 2021

Selected Publications | Full List

*/Preprints/*
	Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts Yue Ma, Yingqing He, Hongfa Wang, Andong Wang, Chenyang Qi, Chengfei Cai, Xiu Li, Zhifeng Li, Heung-Yeung Shum, Wei Liu, Qifeng Chen arXiv preprint:2403.08268. 2024 [paper] [code] [project page]
	MagicStick🪄: Controllable Video Editing via Control Handle Transformations Yue Ma, Xiaodong Cun, Yingqing He, Chenyang Qi, Xintao Wang, Ying Shan, Xiu Li, Qifeng Chen arXiv preprint:2312.03047. 2023 [paper] [code] [project page]
	SimVTP: Simple Video Text Pre-training with Masked Autoencoders Yue Ma, Tianyu Yang, Ying Shan, Xiu Li arXiv preprint:2211.03490. 2022 [paper] [code]
*/Journal/*
	Attentive Snippet Prompting for Video Retrieval Siran Chen, Qinglin Xu, Yue Ma, Yu Qiao, Yali Wang IEEE Transactions on Multimedia (TMM), 2024. [paper] [code]
*/Conference/*
	Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Hengwei Bian, Yatai Ji, Yujiu Yang, Xiu Li Computer Vision and Pattern Recognition (CVPR), 2024 [paper] [code] [project page]
	🕺🕺🕺 Follow-Your-Pose 💃💃💃: Pose-Guided Text-to-Video Generation using Pose-Free Videos Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Ying Shan, Xiu Li, Qifeng Chen The 38th Annual AAAI Conference on Artificial Intelligence (AAAI), 2024 [paper] [code] [project page]
	M-BEV: Masked BEV Perception for Robust Autonomous Driving Siran Chen, Yue Ma, Yu Qiao, Yali Wang The 38th Annual AAAI Conference on Artificial Intelligence (AAAI), 2024 [paper] [code] [project page]
	SemanticAC: Semantics-Assisted Framework for Audio Classification Yicheng Xiao, Yue Ma, Shuyan Li, Hantao Zhou, Ran Liao, Xiu Li (* equal contribution) IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023. [paper] [code]
	Visual Knowledge Graph for Human Action Reasoning in Videos Yue Ma, Yali Wang, Yue Wu, Ziyu Lyu, Siran Chen, Xiu Li, Yu Qiao The 30th ACM International Conference on Multimedia. (ACM MM), 2022. (Oral Presentation) [paper] [code]

Honors & Awards

. . . .

[08/2023] First-Class Scholarship of Tsinghua University.

[12/2022] First-Class Scholarship of SIGS, Tsinghua University.

[03/2022] Tencent Rhino-Bird Research Elite Program, only 72 students in the world admitted to this program.

[09/2020] Scholarship for Academic Excellence of Taiyuan University of Technology.

[06/2019] Excellent Scientific Student of Taiyuan University of Technology.

[09/2019] Scholarship for Academic Excellence of Taiyuan University of Technology.

[09/2018] Excellent Academic Progress Student of Taiyuan University of Technology.

[06/2018] Scholarship for Academic Excellence of Taiyuan University of Technology.

Professional Services

Student Reviewers:
Twelfth International Conference on Learning Representations (ICLR 2024).
The 31th ACM International Conference on Multimedia. (ACM MM 2023).
The 30th Conference on Neural Information Processing Systems. (NeurlPS 2023).

Teaching

2022-2023

Fall

Artificial Intelligence Technology (THU, 85990263-200)