date
type
status
slug
summary
tags
category
icon
password
December 4, 2023 • 2 min read
by Simon Meng, See original
Due to the almost astonishing level of scene reconstruction achieved by 3D Gaussian Splatting, recent papers have begun to focus on text-guided editing. Today, I’ll share three of the most impressive examples 🐶.
1. GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting
https://buaacyw.github.io/gaussian-editor/ (Nanyang Technological University + Tsinghua University + SenseTime)
This editor supports global transfer, addition, or deletion of objects in a Gaussian field guided by text. It even features an interactive UI! It’s open-source, but I haven’t successfully set it up on either Ubuntu or Windows 11; I keep getting the same error. I’ll need to trouble the authors to check out the issue I submitted 😂.
Moreover, it seems the method for field editing is still based on the instruct nerf2nerf approach, so when transferring the entire field in the examples, it still appears somewhat blurry with artifacts, and we can’t expect it to be particularly fast.
2. GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions (Yes, it’s such a coincidence that they share the same name...)
https://gaussianeditor.github.io/ (Huawei, soon to be open-source)
This version supports text-guided editing of specific areas within the Gaussian field. It also appears to use the instruct nerf2nerf method, but it segments regions highly relevant to the text description before editing, which improves speed and stability. The trade-off seems to be that it cannot perform global transfers.
3. Control4D: Efficient 4D Portrait Editing with Text
https://control4darxiv.github.io/ (Tsinghua University, soon to be open-source)
This one is even more impressive, as it not only edits Gaussian fields but also edits 4D (dynamic) Gaussian fields. From what I gathered, it uses a 4D representation called GaussianPlanes, applying plane-based decomposition in 3D space and time to structure Gaussian Splatting better, improving efficiency and robustness in 4D editing. However, the demos focus on portrait editing, so I’m unsure if it can be applied to larger scenes.
- 作者:Simon Shengyu Meng
- 链接:https://shengyu.me//article/AIediting43D-en
- 声明:本文采用 CC BY-NC-SA 4.0 许可协议，转载请注明出处。
相关文章
DreamGaussian: The Stable Diffusion Moment of AIGC 3D Generation
The 2022 Venice - Metaverse Art Annual Exhibition: How Nature Inspires Design
The Basic Principles of ChatGPT
From Hand Modeling to Text Modeling: A Comprehensive Explanation of the Latest AI Algorithms for Generating 3D Models from Text
Andrew Ng's LLM Short Course Notes 1: ChatGPT Prompt Engineering for Developers
Everything about AI Voice Generation (Speech + Music + Lip Sync)