Jipeng Lyu's homepage

jipeng2[at]illinois[dot]edu

Champaign, IL, U.S.

I’m a first-year MSCS student at the University of Illinois Urbana-Champaign, under the supervision of Prof. Yuxiong Wang. I completed my Bachelor of Science degree at Peking University, where I was mentored by Prof. Boxin Shi.

My primary research interests center on the fascinating intersections of computer vision and machine learning, with a specific emphasis on 3D vision. My work involves tasks related to reconstructing real-world scenes and subsequently editing these reconstructed environments.

I have experience in Photometric Stereo and possess relative knowledge in rendering.
Currently, I’m actively exploring NeRF-like methods.

news

Sep 1, 2023	I became an MSCS student @ UIUC. Champaign is a greate place!

selected publications

NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds

Jun-Kun Chen*, Jipeng Lyu*, and Yu-Xiong Wang

In CVPR, 2023

Abs arXiv Bib Website

This paper proposes NeuralEditor that enables neural radiance fields (NeRFs) natively editable for general shape editing tasks. Despite their impressive results on novel-view synthesis, it remains a fundamental challenge for NeRFs to edit the shape of the scene. Our key insight is to exploit the explicit point cloud representation as the underlying structure to construct NeRFs, inspired by the intuitive interpretation of NeRF rendering as a process that projects or "plots" the associated 3D point cloud to a 2D image plane. To this end, NeuralEditor introduces a novel rendering scheme based on deterministic integration within K-D tree-guided density-adaptive voxels, which produces both high-quality rendering results and precise point clouds through optimization. NeuralEditor then performs shape editing via mapping associated points between point clouds. Extensive evaluation shows that NeuralEditor achieves state-of-the-art performance in both shape deformation and scene morphing tasks. Notably, NeuralEditor supports both zero-shot inference and further fine-tuning over the edited scene. Our code, benchmark, and demo video are available at this https URL.
@inproceedings{chen2023neuraleditor, title = {NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds}, author = {Chen*, Jun-Kun and Lyu*, Jipeng and Wang, Yu-Xiong}, booktitle = {CVPR}, year = {2023}, }
NeuralMPS: Non-Lambertian Multispectral Photometric Stereo via Spectral Reflectance Decomposition

Jipeng Lv*, Heng Guo*, Guanying Chen, and 2 more authors

In IJCAI, 2023

Abs arXiv Bib Website

Multispectral photometric stereo(MPS) aims at recovering the surface normal of a scene from a single-shot multispectral image captured under multispectral illuminations. Existing MPS methods adopt the Lambertian reflectance model to make the problem tractable, but it greatly limits their application to real-world surfaces. In this paper, we propose a deep neural network named NeuralMPS to solve the MPS problem under general non-Lambertian spectral reflectances. Specifically, we present a spectral reflectance decomposition(SRD) model to disentangle the spectral reflectance into geometric components and spectral components. With this decomposition, we show that the MPS problem for surfaces with a uniform material is equivalent to the conventional photometric stereo(CPS) with unknown light intensities. In this way, NeuralMPS reduces the difficulty of the non-Lambertian MPS problem by leveraging the well-studied non-Lambertian CPS methods. Experiments on both synthetic and real-world scenes demonstrate the effectiveness of our method.
@inproceedings{lv2022neuralmps, title = {NeuralMPS: Non-Lambertian Multispectral Photometric Stereo via Spectral Reflectance Decomposition}, author = {Lv*, Jipeng and Guo*, Heng and Chen, Guanying and Liang, Jinxiu and Shi, Boxin}, booktitle = {IJCAI}, year = {2023}, }