World models, especially in autonomous driving, are trending and drawing extensive attention due to its
capacity for comprehending driving environments. The established world model holds immense potential
for the generation of high-quality driving videos, and driving policies for safe maneuvering. However,
a critical limitation in relevant research lies in its predominant focus on gaming environments or simulated
settings, thereby lacking the representation of real-world driving scenarios. Therefore, we introduce
DriveDreamer, a pioneering world model entirely derived from real-world driving scenarios. Regarding that
modeling the world in intricate driving scenes entails an overwhelming search space, we propose harnessing
the powerful
世界模型,尤其是自动驾驶模型,由于其理解驾驶环境的容量,正在成为趋势并吸引了广泛的注意力。已建立的世界模型在生成高质量驾驶视频和安全操纵的驾驶政策方面具有巨大的潜力。然而,相关研究的一个关键限制在于其主要关注游戏环境或模拟设置,从而缺乏真实世界驾驶场景的表示。因此,我们推出了 DriveDreamer,这是一款完全源自真实驾驶场景的开创性世界模型。考虑到在复杂的驾驶场景中建模世界需要巨大的搜索空间,我们建议利用强大的扩散模型来构建复杂环境的综合表示。此外,我们介绍了一个两级训练流水线。
The DriveDreamer framework begins with an
DriveDreamer 框架
If you use our work in your research, please cite:
@article{wang2023drive,
title={DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving},
author={Wang, Xiaofeng and Zhu, Zheng and Huang, Guan and Chen, Xinze and Zhu, Jiagang and Lu, Jiwen},
journal={arXiv preprint arXiv:2309.09777},
year={2023}
}