Yue (Jack) Ma

Phd Student

The Hong Kong University of Science and Technology
HongKong.

Email: mayuefighting [at] gmail.com

Biography

I am a first-year Phd student of CSE at The Hong Kong University of Science and Technology (HKUST), under the supervision of Prof. Qifeng Chen. I obtained my M.S. in Computer Science at Tsinghua University in 2024, supervised by Prof. Xiu Li and B.Eng in Computer Science at Taiyuan University of Technology in 2024. I studied in the MMLab@SIAT led by Prof. Dr. Yu Qiao. Currently, I am a research intern at Tencent hunyuan, supervised by Wei Liu (IEEE Fellow) . I was also fortunate to be an internship at Baidu, Tencent AI Lab.

My research interests lie in the intersection of Computer Vision and Machine Learning. From 2021, I started to do some research on video understanding and self-supervised learning. Now, I focus on designing novel applications for image/video generation(Follow-Your-Pose, Follow-Your-Click, Follow-Your-Emoji), video editing(MagicStick). and other downstream AIGC tasks.

Feel free to contact me by email if you are interested in discussing or collaborating with me.

News

[09/2024] One paper are accepted by WACV 2025 !
[08/2024] One paper are accepted by NeurIPS 2024 !
[07/2024] One paper are accepted by Siggraph Asia 2024 !
[07/2024] One paper are accepted by ACM MM 2024 !
[06/2024] Excited to release Follow-Your-Emoji, a freestyle portrait animation framework. Enjoy FollowYourEmoji quickly !
[03/2024] Excited to release Follow-Your-Click, the first framework to achieve regional image animation. Enjoy the fun of FollowYourClick !
[01/2024] One paper are accepted by CVPR 2024 !
[12/2023] Two paper are accepted by AAAI 2024 !
[11/2023] Excited to release MagicStick, the first unified framework to modify video properties leveraging the keyframe transformations on the extracted internal control signals.
[10/2023] One paper is accepted by TMM 2024 !
[09/2023] I am serving as a Reviewer for ICLR 2024 !
[07/2023] I start to cooperate with Qifeng Chen closely in HKUST !
[06/2023] I end my research intership in Tencent AI Lab !
[06/2023] I am serving as a Reviewer for NeurIPS 2023 !
[05/2023] I am serving as a Reviewer for ACM MM 2023 !
[04/2023] Excited to release Follow-Your-Pose framework, generating the character videos from pose and text description. Enjoy the fun of FollowYourPose !
[03/2023] Two paper are submitted to ICCV 2023 !
[02/2023] One paper SemanticAC about Audio Classification was accepted by ICASSP 2023 !
[12/2022] Awarded First-Class Scholarship of Tsinghua Shenzhen International Graduate School !
[11/2022] One paper about video-text pretraining with masked autoencoder is submitted to CVPR 2023 !
[10/2022] One paper about semantics-assisted framework for audio classification is submitted to ICASSP 2023 !
[09/2022] Attending ACM MM 2022, Welcome to chat!
[09/2022] My first paper has been accepted for Oral Presentation on ACM MM 2022 (Top 5%) !
[05/2022] Joined Tencent AI Lab as research intern !
[03/2022] Awarded Tencent Rhino-Bird Research Elite Program from Tencent ! Only 72 students in the world admitted to this program !
[07/2021] Joined CUHK@MMLab in ShenZhen as research intern !
[06/2021] Recommended to Tsinghua University towards the MSc degree !
[06/2021] Graduated from Taiyuan University of Technology !
[01/2021] Joined Baidu Research as research intern. Started doing research on Computer Vision !

Industrial Experience

Tencent Hunyuan

Dec. 2023 - Present, Tencent Hunyuan, ShenZhen, China

worked with Dr. Wei Liu

Topic: Video Generation (Follow Family) and Video Editing

Tencent AI Lab

May. 2022 - July. 2023, Tencent AI Lab, Visual Center, ShenZhen, China

worked with Dr. Tianyu Yang, Dr. Xiaodong Cun and Dr. Xintao Wang

Topic: Video Generation and Video Editing

Baidu VIS

Jan. 2021 - July. 2021, Baidu, Visual group, ShenZhen, China

worked with Zhikang Zou

Topic: 3D Object Detection, 3D Image Understanding

Education & Visiting

	The Hong Kong University of Science and Technology, Hong Kong PhD Student in Visual Intelligence Lab, HKUST Advisor: Prof. Qifeng Chen Sep. 2024 - Future
	The Hong Kong University of Science and Technology, Hong Kong Research Assistant in Visual Intelligence Lab, HKUST Advisor: Prof. Qifeng Chen July. 2023 - Nov. 2023
	Tsinghua University, China Master of Engineering in Electronic Information Advisor: Prof.Xiu Li Sep. 2021 - Jun. 2024
	University of Chinese Academy of Sciences, China Research Assistant in Multimedia Laboratory, SIAT, CAS (MMLab@SIAT) Advisor: Prof. Yu Qiao and Dr. Yali Wang Jul. 2021 - Apr. 2022
	Taiyuan University of technology, China Bachelor of Engineering in Computer Science Sep. 2017 - Jun. 2021

Selected Publications | Full List

*/Preprints/*
	Follow-Your-Pose v2: Multiple-Condition Guided Character Image Animation for Stable Pose Control Jingyun Xue, Hongfa Wang, Qi Tian, Yue Ma, Andong Wang, Zhiyuan Zhao, Shaobo Min, Wenzhe Zhao, Kaihao Zhang, Heung-Yeung Shum, Wei Liu, Mengyang Liu, Wenhan Luo arXiv preprint:2406.03035. 2024 [paper] [code] [project page]
	Follow-Your-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generation Qihua Chen, Yue Ma, Hongfa Wang, Junkun Yuan, Wenzhe Zhao, Qi Tian, Hongmei Wang, Shaobo Min, Qifeng Chen, Wei Liu arXiv preprint:2409.01055. 2024 [paper] [code] [project page]
	MultiBooth: Towards Generating All Your Concepts in an Image from Text Chenyang Zhu, Kai Li, Yue Ma, Chunming He, Xiu Li arXiv preprint:2404.14239. 2024 [paper] [code] [project page]
	Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts Yue Ma, Yingqing He, Hongfa Wang, Andong Wang, Chenyang Qi, Chengfei Cai, Xiu Li, Zhifeng Li, Heung-Yeung Shum, Wei Liu, Qifeng Chen arXiv preprint:2403.08268. 2024 [paper] [code] [project page]
	SimVTP: Simple Video Text Pre-training with Masked Autoencoders Yue Ma, Tianyu Yang, Ying Shan, Xiu Li arXiv preprint:2211.03490. 2022 [paper] [code]
*/Journal/*
	Attentive Snippet Prompting for Video Retrieval Siran Chen, Qinglin Xu, Yue Ma, Yu Qiao, Yali Wang IEEE Transactions on Multimedia (TMM), 2024. [paper] [code]
*/Conference/*
	MagicStick🪄: Controllable Video Editing via Control Handle Transformations Yue Ma, Xiaodong Cun, Yingqing He, Chenyang Qi, Xintao Wang, Ying Shan, Xiu Li, Qifeng Chen IEEE /CVF Winter Conference on Applications of Computer Vision (WACV), 2025 [paper] [code] [project page]
	Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation Yue Ma, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yeung Shum, Wei Liu, Qifeng Chen The ACM Special Interest Group for Computer Graphics and Interactive Techniques(Siggraph Asia) 2024 [paper] [code] [project page]
	COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editing Jiangshan Wang, Yue Ma, Jiayi Guo, Yicheng Xiao, Gao Huang, Xiu Li Conference on Neural Information Processing Systems(NeurIPS). 2024* [paper] [code] [project page]
	Freehand Sketch Generation from Mechanical Components Zhichao Liao, Di Huang, Heming Fang, Yue Ma, Fengyuan Piao, Xinghui Li, Long Zeng, Pingfa Feng The 32th ACM International Conference on Multimedia. (ACM MM), 2024. [paper] [code] [project page]
	Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Hengwei Bian, Yatai Ji, Yujiu Yang, Xiu Li IEEE /CVF Computer Vision and Pattern Recognition (CVPR), 2024 [paper] [code] [project page]
	🕺🕺🕺 Follow-Your-Pose 💃💃💃: Pose-Guided Text-to-Video Generation using Pose-Free Videos Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Ying Shan, Xiu Li, Qifeng Chen The 38th Annual AAAI Conference on Artificial Intelligence (AAAI), 2024 [paper] [code] [project page]
	M-BEV: Masked BEV Perception for Robust Autonomous Driving Siran Chen, Yue Ma, Yu Qiao, Yali Wang The 38th Annual AAAI Conference on Artificial Intelligence (AAAI), 2024 [paper] [code] [project page]
	SemanticAC: Semantics-Assisted Framework for Audio Classification Yicheng Xiao, Yue Ma, Shuyan Li, Hantao Zhou, Ran Liao, Xiu Li (* equal contribution) IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023. [paper] [code]
	Visual Knowledge Graph for Human Action Reasoning in Videos Yue Ma, Yali Wang, Yue Wu, Ziyu Lyu, Siran Chen, Xiu Li, Yu Qiao The 30th ACM International Conference on Multimedia. (ACM MM), 2022. (Oral Presentation) [paper] [code]

Honors & Awards

. . . .

[06/2024] Outstanding graduates student of Beijing.

[08/2023] First-Class Scholarship of Tsinghua University.

[12/2022] First-Class Scholarship of SIGS, Tsinghua University.

[03/2022] Tencent Rhino-Bird Research Elite Program, only 72 students in the world admitted to this program.

[09/2020] Scholarship for Academic Excellence of Taiyuan University of Technology.

[06/2019] Excellent Scientific Student of Taiyuan University of Technology.

[09/2019] Scholarship for Academic Excellence of Taiyuan University of Technology.

[09/2018] Excellent Academic Progress Student of Taiyuan University of Technology.

[06/2018] Scholarship for Academic Excellence of Taiyuan University of Technology.

Professional Services

Student Reviewers:
Computer Vision and Pattern Recognition (CVPR)
International Conference on Computer Vision (ICCV)
European Conference on Computer Vision (ECCV)
Conference and Workshop on Neural Information Processing Systems (NeurIPS)
International Conference on Learning Representations (ICLR)
AAAI Conference on Artificial Intelligence (AAAI)
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

Teaching

2022-2023

Fall

Artificial Intelligence Technology (THU, 85990263-200)