Applying DDP for Semantic Segmentation
应用 DDP 进行语义分割

Our segmentation code is developed on top of MMSegmentation v0.20.2.
我们的分割代码是在 MMSegmentation v0.20.2 基础上开发的。

For details please see DDP
详情请参见 DDP

If this work is helpful for your research, please consider citing the following BibTeX entry.
如果该作品对您的研究有帮助，请考虑引用以下 BibTeX 条目。

@article{ji2023ddp,
  title={DDP: Diffusion Model for Dense Visual Prediction},
  author={Ji, Yuanfeng and Chen, Zhe and Xie, Enze and Hong, Lanqing and Liu, Xihui and Liu, Zhaoqiang and Lu, Tong and Li, Zhenguo and Luo, Ping},
  journal={arXiv preprint arXiv:2303.17559},
  year={2023}
}

Installation 安装

The code is based on the MMSegmentation v0.29.0+
代码基于 MMSegmentation v0.29.0+ 版本

# recommended to create a new environment with torch1.12 + cuda11.6
pip install torch==1.12.0+cu116 torchvision==0.13.0+cu116 torchaudio==0.12.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install mmcv-full==1.6.2 -f https://download.openmmlab.com/mmcv/dist/cu116/torch1.12.0/index.html
cd DDP/segmentation/
pip install -v -e .

Data Preparation 数据准备

Preparing ADE20K/Cityscapes according to the guidelines in MMSegmentation.
根据 MMSegmentation 中的指导原则准备 ADE20K/Cityscapes。

Results and Models
结果和模型

Cityscapes val (step 3) 城市景观价值（第 3 步）

Backbone 骨干网	Lr schd	Crop Size 作物大小	mIoU (SS/MS)	#Param	Config 配置	Download 下载
ConvNext-T	160K	512x1024	82.60/83.15	40M	config 配置	ckpt \| log
ConvNext-S	160K	512x1024	82.69/83.58	62M	config 配置	ckpt \| log
ConvNext-B	160K	512x1024	82.78/83.49	100M	config 配置	ckpt \| log
ConvNext-L	160K	512x1024	83.21/83.92	209M	config 配置	ckpt \| log

Cityscapes val (with self-aligned denoising, single scale)
城市景观值（自对齐去噪、单一比例尺）

Backbone 骨干网	Lr schd	Crop Size 作物大小	mIoU (step 1/3/10) 联检组（步骤 1/3/10）	#Param	Config 配置	Download 下载
ConvNext-T	5K	512x1024	82.3/82.6/82.6	40M	config 配置	ckpt \| log
ConvNext-L	5K	512x1024	83.0/83.2/83.2	209M	config 配置	ckpt \| log

ADE20k val (step 3) ADE20k val（步骤 3）

Backbone 骨干网	Lr schd	Crop Size 作物大小	mIoU (SS/MS)	#Param	Config 配置	Download 下载
Swin-T	160K	512x512	47.0/47.8	40M	config 配置	ckpt \| log
Swin-S	160K	512x512	48.7/49.7	61M	config 配置	ckpt \| log
Swin-B 斯温-B	160K	512x512	49.4/50.8	99M	config 配置	ckpt \| log
Swin-L	160K	512x512	53.2/54.4	207M	config 配置	ckpt \| log

Training 培训

Multi-gpu training 多 GPU 培训

bash tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM}

For example, To train DDP-ConvNext-L on cityscapes with 4 gpus run:
例如，使用 4 个 GPU 在城市景观上训练 DDP-ConvNext-L 运行：

bash tools/dist_train.sh configs/cityscapes/ddp_convnext_l_4x4_512x1024_160k_cityscapes.py 4

Self-aligned denoising 自对齐去噪

bash tools/dist_train.sh configs/cityscapes/ddp_convnext_t_4x4_512x1024_5k_cityscapes_aligned.py 4

Evaluation 评估

Single-gpu testing 单 GPU 测试

python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --eval mIoU

Multi-gpu testing 多 GPU 测试

bash tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM} --eval mIoU

For example, To evaluate DDP-ConvNext-T on cityscapes val on a single node with 4 gpus run:
例如，要评估 DDP-ConvNext-T 在 4 个 GPU 单节点上的城市景观值，请运行

bash tools/dist_test.sh configs/cityscapes/ddp_convnext_t_4x4_512x1024_160k_cityscapes.py ckpts/ddp_convnext_t_4x4_512x1024_160k_cityscapes.pth 4 --eval mIoU

This should give the below results. Note that the results may vary a little on different machines due to the randomness of the diffusion modeling.
结果如下。请注意，由于扩散建模的随机性，在不同机器上得出的结果可能会略有不同。

Summary:

+-------+-------+-------+
|  aAcc |  mIoU |  mAcc |
+-------+-------+-------+
| 96.85 | 82.71 | 88.76 |
+-------+-------+-------+

Image Demo 图像演示

To inference a single image like this:
像这样推理一张图片

python image_demo.py ${IMAGE_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} ${PRED_FILE} --device cuda:0 --palette ${PALETTE_FILE}

For example, the result will be saved in resources/demo_pred.png by running:
例如，运行后结果将保存在 resources/demo_pred.png 中：

python image_demo.py resources/demo.png configs/ddp/ddp_convnext_t_4x4_512x1024_160k_cityscapes.py ckpts/ddp_convnext_t_4x4_512x1024_160k_cityscapes.pth resources/demo_pred.png --device cuda:0 --palette cityscapes

Applying DDP for Semantic Segmentation应用 DDP 进行语义分割