Zhenzhen (Jen) Weng

I am a final year Ph.D student in Computational and Mathematical Engineering (ICME) at Stanford University where I am advised by Prof. Serena Yeung.

I am broadly interested in 3D computer vision. Specifically, I am interested in human-centric 3D perception and generative models. I have interned at Waymo Research in 2022 where I worked on efficient human 3D pose estimation from LiDAR data. In 2023, I interned at Adobe Research and worked on generalizable single-view human NeRF prediction.

Prior to my Ph.D, I received B.S. in Computer Science and B.S. in Mathematics from Carnegie Mellon University. I also previously worked for a fund manager on the East Coast.

I will be graduating in June 2024 and I am looking for research scientist roles in 3D computer vision.

Email  /  Linkedin  /  Twitter  /  Google Scholar

Oct, 2023: 1 paper accepted to 3DV 2024. See you in Davos, Switzerland!

June, 2023: Attended CVPR 2023 in Vancouver, Canada.

Feb, 2023: 2 papers accepted to CVPR 2023.

Jan, 2023: Joining Adobe Research as a Research Scientist Intern this summer.

Reconstructing 3D humans from a single image has been extensively investigated. However, existing approaches often fall short on capturing fine geometry and appearance details, hallucinating occluded parts with plausible details, and achieving generalization across unseen and in-the-wild datasets. We present \Ours{}, a diffusion-guided feed-forward model that predicts the implicit field of a human from a single image. Leveraging the power of the state-of-the-art reconstruction model (i.e., LRM) and generative model (i.e Stable Diffusion), our method is able to capture human without any template prior, e.g., SMPL, and effectively enhance occluded parts with rich and realistic details.

Recent text-to-image generative models such as Stable Diffusion often struggle to preserve plausible human structure in the generations. We propose Diffusion model with Human Pose Correction (Diffusion-HPC), a method that generates photo-realistic images with plausible posed humans by injecting prior knowledge about human body structure. The generated image-mesh pairs are well-suited for downstream human mesh recovery task.

We present ZeroAvatar, a method that introduces the explicit 3D human body prior to the optimization process. We show that ZeroAvatar significantly enhances the robustness and 3D consistency of optimization-based image-to-3D avatar generation, outperforming existing zero-shot image-to-3D methods.

We propose GC-KPL - Geometry Consistency inspired Key Point Leaning. By training on the large WOD training set without any annotated keypoints, we attain reasonable performance as compared to the fully supervised approach. Further, the backbone benefits from the unsupervised training and is useful in downstream fewshot learning of keypoints, where fine-tuning on only 10 percent of the labeled training data gives comparable performance to fine-tuning on the entire set.

We aim to bridge the gap between monocular HMR and multi-view MoCap systems by leveraging information shared across multiple video instances of the same action. We introduce the Neural Motion (NeMo) field. It is optimized to represent the underlying 3D motions across a set of videos of the same action.

We propose Domain Adaptive 3D Pose Augmentation (DAPA), a data augmentation method that combines the strength of methods based on synthetic datasets by getting direct supervision from the synthesized meshes.

We propose a holistically trainable model that perceives the 3D scene from a single RGB image, estimates the camera pose and the room layout, and reconstructs both human body and object meshes.

We propose a method that can perform unsupervised discovery of long-tail categories in instance segmentation, through learning instance embeddings of masked regions.

We introduce the challenge of improving slice-specific performance without damaging the overall model quality, and proposed the first programming abstraction and machine learning model to support these actions.

We introduced weak supervision heuristics as a methodology to infer complex objects and situations by combining simpler outputs from current, state-of-the art object detectors.

Research Scientist Intern @ Adobe Research, San Jose, CA, Jun - Sept, 2023

Research Intern (Perception) @ Waymo, Mountain View, CA, Jun - Nov, 2022

Machine Learning Engineer @ VMware, Palo Alto, CA, Jun - Sept, 2019

Research Engineer @ AQR, Greenwich, CT, 2016 - 2018

Webpage template and source code from Jon Barron.