About me

• I am currently working as an assistant researcher, collaborating with Prof. [Zhiming Zheng], Academician of the Chinese Academy of Sciences（中国科学院院士） and Prof. [Wenjun Wu], at School of Artificial Intelligence, Beihang University. I am also a Joint Appointment Researcher at The Institute of AI (TeleAI), China Telecom, Beijing, P.R. China.

• I am now also working as the chief young scientist of Psyche AI Inc（深锶科技）, fosuing on developing advanced AI algorithms for avatars, including avatar driving, avatar generation, and avatar reconsctrution.

• I pursued my PhD from 2019 to 2024 across three prestigious institutions: Carnegie Mellon University（卡内基梅隆大学）, The Hong Kong University of Science and Technology（香港科技大学）, and Renmin University of China（中国人民大学）, studying Computer Science (计算机科学). At CMU, I had the privilege of working under the guidance of Prof. Min Xu. My time at HKUST was spent collaborating with Prof. Kai Chen. While at RUC, I worked closely with Prof. Jun He and Prof. Hongyan Liu from Tsinghua University. My research interests lie in the areas of multi-modal LLMs, avatars, and embodied AI.Please feel free to contact me by email (fanzhaoxinruc@gmail.com or zhaoxinf@buaa.edu.cn), if you are interested in my research.

• If you are interested in becoming my student or a close collaborator, please answer the following questions before contacting me via email:

  • Firstly, familiarize yourself with my research area by reading my recent papers and visiting our research group’s webpage. Research requires passion, and lack of interest may hinder persistence.
  • If you are applying for a direct PhD or a combined Master’s and PhD program, it implies that you see academia as a career path. Please decide after understanding the challenges and rewards of an academic life.
  • When applying for a Master’s program, I value excellent undergraduate grades, a solid foundation in mathematics, proficient programming skills, and good English proficiency.
  • I hope you are optimistic and proactive, with a resilient spirit, clear logical thinking, and good teamwork skills.
  • After considering these points, if you still wish to choose me as your mentor, please send your resume via email, including personal details, academic records, work or internship experiences, research experience, publications, and any other relevant information. I will respond as soon as possible.

News!

• July/1/2024, Glad to annouce that MLPHand is accepted to ECCV 2024 main track.
• April/7/2024, Excited to annouce that 5 papers are accepted to ICMR 2024.
• March/22/2024, Congrats to Han Sun for his first paper being accepted to IEEE Transactions on Instrumentation & Measurement.
• Feb/27/2024, Glad to annouce that SyncTalk is accepted to CVPR 2024 main track.
• Dec/9/2023, Our recent paper Everything2Motion is accepted to AAAI 2024 main track and the paper FurPE is accepted to AAAI 2024 workshop. Congrats to all authors.
• Nov/17/2023, Congrats to Yixing Lu for his first paper being accepted to International Conference on Multimedia Modeling
• Sep/21/2023, We are awarded the second prize in the National Challenge Cup! I together with Professor Li Ronghua serve as the advisors of this project.
• July/26/2023, I am thrilled to announce that our latest work, SelfTalk, has been accepted by ACM MM 2023!
• Two papers D-IF and Emo-Talk are accepted by ICCV 2023, one of the best conference in computer vision.
• Excited to annouce that one of our papers has been accepted by signal processing, IF=4.729
• Excited to annouce that one of my papers has been accepted by IJCAI 2023.
• I am honored to have returned to Carnegie Mellon University as a visiting scholar.
• One of my papers has been accepted for presentation at CVPR 2023, which is considered one of the most prestigious computer vision conferences in the world.
• I am thrilled to have had one of my papers accepted for presentation at ICRA 2023, which is a leading robotics conference.
• My research has been recognized at ECCV Workshop 2022, PRCV 2022, and ECCV 2022, all of which are highly-regarded international conferences in the field of computer vision.
• I am proud to have had a paper accepted for publication in the ACM Computing Surveys journal, which is widely considered one of the best ACM journals with an impact factor of 14.324.
• The virtual actor An-Ruohan of Psyai, which I helped to develop, performs her show daily on bilibili.
• My research has also been recognized at ICIP 2022, which is a top-tier computer vision conference with a long-standing reputation for excellence.
• I am excited to announce that one of my papers has been accepted for presentation at AAAI 2022, which is a leading artificial intelligence conference.

Working Experience

• Researcher, Psyche AI Inc (2021 - Present)
  • As a researcher at Psyche AI Inc, I am responsible for leading and overseeing various projects related to human body reconstruction, talking head technology, and 3D foundation models. In this role, I contribute my expertise in the development of innovative AI solutions that advance the field of computer vision and digital avatars.
• Remote Algorithm Researcher, Xreal (2021 - 2023)
  • As a Remote Algorithm Researcher at Xreal, my primary focus is on the research and development of algorithms for 6D object pose estimation, point cloud completion, and 3D foundation models. Working remotely, I collaborate with a team of researchers and engineers to design and implement cutting-edge algorithms that enhance the accuracy and efficiency of computer vision systems.
• Intern, Bytedance Inc (2020 - 2021)
  • During my internship at Bytedance Inc, I gained valuable experience in the field of computer vision, specifically in the areas of 6D object pose estimation, 3D object detection, and 3D object tracking. As an intern, I actively contributed to the development of algorithms and systems, working closely with experienced professionals in the industry.

Selected Publications

• MLPHand: Real Time Multi-View 3D Hand Mesh Reconstruction via MLP Modeling [paper] [code]
  Jian Yang, Jiakun Li, Guoming Li, Zhen Shen, Huai-Yu Wu, Zhaoxin Fan(corresponding author)
  European Conference on Computer Vision, ECCV 2024

• SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis [paper][code]
  Ziqiao Peng, Wentao Hu, Yue Shi, Xiangyu Zhu, Xiaomei Zhang, Hao Zhao, Jun He, Hongyan Liu, Zhaoxin Fan (corresponding author).
  IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2024

• MonoSIM: Simulating Learning Behaviors of Heterogeneous Point Cloud Object Detectors for Monocular 3D Object Detection [paper] [code]
  Han Sun, Zhaoxin Fan (equal contribution), Zhenbo Song, Zhicheng Wang, Kejian Wu, and Jianfeng Lu.
  IEEE Transactions on Instrumentation & Measurement, TIM 2024

• Everything2Motion: Synchronizing Diverse Inputs via a Unified Framework for Human Motion Synthesis[paper]
  Zhaoxin Fan, Longbin Li, Pengxin Xu, Fan Shen, kai Chen.
  Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024

• EmoTalk: Speech-driven emotional disentanglement for 3D face animation [paper] [code]
  Ziqiao Peng, Haoyu Wu, Zhenbo Song, Hao Xu, Xiangyu Zhu, Hongyan Liu, Jun He, Zhaoxin Fan(corresponding author).
  International Conference on Computer Vision, ICCV 2023

• D-IF: Uncertainty-aware Human Digitization via Implicit Distribution Field [paper][code]
  Xueting Yang, Yihao Luo, Yuliang Xiu, Wei Wang, Hao Xu, Zhaoxin Fan(corresponding author).
  International Conference on Computer Vision, ICCV 2023

• SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces [paper][code]
  Ziqiao Peng, Yihao Luo, Yue Shi, Hao Xu, Xiangyu Zhu, Hongyan Liu, Jun He, Zhaoxin Fan(corresponding author).
  ACM International Conference on Multimedia, ACM MM 2023

• Reconstruction-Aware Prior Distillation for Semi-supervised Point Cloud Completion [paper]
  Zhaoxin Fan, Yulin He, Zhicheng Wang, Kejian Wu, Hongyan Liu and Jun He
  International Joint Conference on Artificial Intelligence, IJCAI 2023

• Robust Single Image Reflection Removal Against Adversarial Attacks [paper]
  Zhenbo Song, Zhenyuan Zhang, Kaihao Zhang, Wenhan Luo, Zhaoxin Fan, Wenqi Ren, Jianfeng Lu
  IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2023

• Object Level Depth Reconstruction for Category Level 6D Object Pose Estimation From Monocular RGB Image [paper][code]
  Zhaoxin Fan, Zhenbo Song, Jian Xu, Zhicheng Wang, Kejian Wu, Hongyan Liu and Jun He
  European Conference on Computer Vision, ECCV 2022

• SVT-Net: Super Light-Weight Sparse Voxel Transformer for Large Scale Place Recognition [paper][code]
  Zhaoxin Fan, Zhenbo Song, Zhiwu Lu, Hongyan Liu, Jun He, and Xiaoyong Du
  36th AAAI Conference on Artificial Intelligence, AAAI 2022

• Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview [paper]
  Zhaoxin Fan, Yazhi Zhu, Yulin He, Qi Sun, Hongyan Liu, and Jun He
  ACM Computing Surveys, CSUR, 2022.

• GIDP: Learning a Good Initialization and Inducing Descriptor Post-enhancing for Large-scale Place Recognition [paper]
  Zhaoxin Fan, Zhenbo Song, Hongyan Liu, and Jun He
  International Conference on Robotics and Automation, ICRA 2023.

• SRNet: A 3D Scene Recognition Network using Static Graph and Dense Semantic Fusion [paper]
  Zhaoxin Fan, Hongyan Liu, Jun He, Qi Sun, and Xiaoyong Du
  Computer Graphics Forum, CGF 2020

• A Graph‐based One‐Shot Learning Method for Point Cloud Recognition [paper]
  Zhaoxin Fan, Hongyan Liu, Jun He, Qi Sun, and Xiaoyong Du
  Computer Graphics Forum, CGF 2020