这是用户在 2025-6-8 1:18 为 https://github.com/dailenson/SDT?tab=readme-ov-file 保存的双语快照页面,由 沉浸式翻译 提供双语支持。了解如何保存?
Skip to content
Owner avatar SDT Public

This repository is the official implementation of Disentangling Writer and Character Styles for Handwriting Generation (CVPR 2023)

License

Public repository
Open in github.dev Open in a new github.dev tab Open in codespace

dailenson/SDT

Add file

Add file

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Apr 20, 2025
339aa21 · Apr 20, 2025

History

56 Commits
Apr 20, 2025
Nov 26, 2024
Nov 26, 2024
Apr 19, 2024
Jun 27, 2023
Nov 26, 2024
Jan 7, 2024
Jun 27, 2023
Nov 26, 2024
Jun 27, 2023
Aug 26, 2024
Nov 26, 2024
Jun 27, 2023
Dec 17, 2023
Jun 27, 2023
Jan 7, 2024

Repository files navigation

MIT LICENSE python 3.8

🔥 Disentangling Writer and Character Styles for Handwriting Generation
🔥 手写体生成中的书写者与字符风格解耦

ArXiv | Poster | Video | Project

📢 Introduction

  • The proposed style-disentangled Transformer (SDT) generates online handwritings with conditional content and style.
    提出的风格解耦 Transformer(SDT)能够根据条件内容与风格生成在线手写体。
  • Existing RNN-based methods mainly focus on capturing a person’s overall writing style, neglecting subtle style inconsistencies between characters written by the same person. In light of this, SDT disentangles the writer-wise and character-wise style representations from individual handwriting samples for enhancing imitation performance.
    现有基于 RNN 的方法主要关注捕捉个人的整体书写风格,忽略了同一个人书写不同字符时存在的细微风格不一致性。针对这一问题,SDT 从单个手写样本中解耦了作者级和字符级风格表示,以增强模仿性能。
  • We extend SDT and introduce an offline-to-offline framework for improving the generation quality of offline Chinese handwritings.
    我们扩展了 SDT,并引入了一个离线到在线的框架,用于提高离线中文手写体的生成质量。

Overview of our SDT

Three samples of online characters with writing orders
带有书写顺序的在线字符样本三例

📅 News  📅 新闻

  • [2024/11/26] 🎉🎉🎉 Release of the implementations of Content Score and Style Score.
    [2024/11/26] 🎉🎉🎉 Content Score 和 Style Score 的实现已发布。
  • [2024/07/01] 🎉🎉🎉 A new state-of-the-art method for handwritten text generation, named One-DM, is accepted by ECCV 2024.
    [2024/07/01] 🎉🎉🎉 一种名为 One-DM 的新手写文本生成尖端方法已被 ECCV 2024 录用。
  • [2024/01/07] Add a tutorial and code for synthesizing handwriting with user-customized styles, more information can be found here.
    [2024/01/07] 添加了用于合成用户自定义风格手写体的教程和代码,更多信息可以在这里找到。
  • [2023/12/15] 🎉🎉🎉 This work is reported by a top bilibili video blogger with 2.7 million followers and received nearly one million views.
    [2023/12/15] 🎉🎉🎉 该工作被一位拥有 270 万粉丝的顶级 bilibili 视频博主报道,并获得了近 100 万次观看。
  • [2023/10/10] The author is invited to give a talk (in Chinese) by CSIG (China Society of Image and Graphics).
    [2023/10/10] 作者受邀由中国图象图形学学会(CSIG)进行了一场中文演讲。
  • [2023/06/14] This work is reported by Synced (机器之心).
    [2023/06/14] 本工作由 Synced(机器之心)报道。
  • [2023/04/12] Initial release of the datasets, pre-trained models, training and testing codes.
    [2023/04/12] 数据集、预训练模型、训练和测试代码的初始发布。
  • [2023/02/28] 🎉🎉🎉 Our SDT is accepted by CVPR 2023.
    [2023/02/28] 🎉🎉🎉 我们的 SDT 被 CVPR 2023 录用。

📺 Handwriting generation results
📺 手写生成结果

  • Online Chinese handwriting generation
    在线中文手写生成
    online Chinese

  • Applications to various scripts
    应用于多种脚本
    other scripts

  • Extension on offline Chinese handwriting generation
    扩展离线中文手写生成
    offline Chinese

🔨 Requirements  🔨 依赖要求

conda create -n sdt python=3.8 -y
conda activate sdt
# install all dependencies
conda env create -f environment.yml

📂 Folder Structure  📂 文件夹结构

SDT/
│
├── train.py - main script to start training
├── test.py - generate characters via trained model
├── evaluate.py - evaluation of generated samples
│
├── configs/*.yml - holds configuration for training
├── parse_config.py - class to handle config file
│
├── data_loader/ - anything about data loading goes here
│   └── loader.py
│
├── model_zoo/ - pre-trained content encoder model
│
├── data/ - default directory for storing experimental datasets
│
├── model/ - networks, models and losses
│   ├── encoder.py
│   ├── gmm.py
│   ├── loss.py
│   ├── model.py
│   └── transformer.py
│
├── saved/
│   ├── models/ - trained models are saved here
│   ├── tborad/ - tensorboard visualization
│   └── samples/ - visualization samples in the training process
│
├── trainer/ - trainers
│   └── trainer.py
│  
└── utils/ - small utility functions
    ├── util.py
    └── logger.py - set log dir for tensorboard and logging output

💿 Datasets  💿 数据集

We provide Chinese, Japanese and English datasets in Google Drive | Baidu Netdisk PW:xu9u. Please download these datasets, uzip them and move the extracted files to /data.
我们提供了中文、日文和英文数据集,存储在 Google Drive | Baidu Netdisk PW:xu9u。请下载这些数据集,解压并将解压后的文件移动到/data 目录。

🍔 Pre-trained model  🍔 预训练模型

Model Google Drive Baidu Netdisk
Well-trained SDT Google Drive Baidu Netdisk
Content encoder  内容编码器 Google Drive Baidu Netdisk
Content Score  内容评分 Google Drive Baidu Netdisk
Style Score Google Drive Baidu Netdisk

Note: Please download these weights, and move them to /model_zoo.
注意:请下载这些权重,并将它们移动到 /model_zoo。

🚀 Training & Test  🚀 训练 & 测试

Training  训练

  • To train the SDT on the Chinese dataset, run this command:
    要在中文数据集上训练 SDT,请运行此命令:
python train.py --cfg configs/CHINESE_CASIA.yml --log Chinese_log
  • To train the SDT on the Japanese dataset, run this command:
    训练 SDT 模型使用日语数据集,运行以下命令:
python train.py --cfg configs/Japanese_TUATHANDS.yml --log Japanese_log
  • To train the SDT on the English dataset, run this command:
    训练 SDT 模型使用英语数据集,运行以下命令:
python train.py --cfg configs/English_CASIA.yml --log English_log

Qualitative Test  定性测试

  • To generate online Chinese handwritings with our SDT, run this command:
    要使用我们的 SDT 模型生成在线中文手写体,运行以下命令:
python test.py --pretrained_model checkpoint_path --store_type online --sample_size 500 --dir Generated/Chinese
  • To generate offline Chinese handwriting images with our SDT, run this command:
    使用我们的 SDT 生成离线中文手写图像,运行以下命令:
python test.py --pretrained_model checkpoint_path --store_type offline --sample_size 500 --dir Generated_img/Chinese
  • To generate online Japanese handwritings with our SDT, run this command:
    使用我们的 SDT 生成在线日文手写,运行以下命令:
python test.py --pretrained_model checkpoint_path --store_type online --sample_size 500 --dir Generated/Japanese
  • To generate offline Japanese handwriting images with our SDT, run this command:
    使用我们的 SDT 生成离线日文手写图像,运行以下命令:
python test.py --pretrained_model checkpoint_path --store_type offline --sample_size 500 --dir Generated_img/Japanese
  • To generate online English handwritings with our SDT, run this command:
    使用我们的 SDT 生成在线英文手写,运行以下命令:
python test.py --pretrained_model checkpoint_path --store_type online --sample_size 500 --dir Generated/English
  • To generate offline English handwriting images with our SDT, run this command:
    要使用我们的 SDT 生成离线英文手写图像,请运行以下命令:
python test.py --pretrained_model checkpoint_path --store_type offline --sample_size 500 --dir Generated_img/English

Quantitative Evaluation  定量评估

  • To evaluate the generated handwritings, you need to set data_path to the path of the generated handwritings (e.g., Generated/Chinese), and run this command:
    要评估生成的手写体,您需要将 data_path 设置为生成手写体的路径(例如,Generated/Chinese),然后运行以下命令:
python evaluate.py --data_path Generated/Chinese --metric DTW
  • To calculate the Content Score of generated handwritings, you need to set data_path to the path of the generated handwritings (e.g., Generated/Chinese), and run this command:
    要计算生成手写体的内容得分,您需要将 data_path 设置为生成手写体的路径(例如,Generated/Chinese),然后运行以下命令:
python evaluate.py --data_path Generated/Chinese --metric Content_score --pretrained_model model_zoo/chinese_content_iter30k_acc95.pth
  • To calculate the Style Score of generated handwritings, you need to set data_path to the path of the generated handwriting images (e.g., Generated_img/Chinese), and run this command:
python evaluate.py --data_path Generated_img/Chinese --metric Style_score --pretrained_model models_zoo/chinese_style_iter60k_acc999.pth

🏰 Practical Application  🏰 实际应用

We are delighted to discover that P0etry-rain has proposed a pipeline that involves initially converting the generated results by our SDT to TTF format, followed by the development of software to enable flexible adjustments in spacing between paragraphs, lines, and characters. Below, we present TTF files, software interface and the printed results. More details can be seen in #78.
我们很高兴地发现 P0etry-rain 提出了一个流程,该流程首先将我们 SDT 生成的结果转换为 TTF 格式,然后开发软件以实现段落、行和字符间距的灵活调整。下面我们展示了 TTF 文件、软件界面和打印结果。更多细节请参见 #78。

  • TTF File  TTF 文件 SVG

  • Software Interface  软件界面 Interface

  • Printed Results  打印结果 Result

❤️ Citation  ❤️ 引用

If you find our work inspiring or use our codebase in your research, please cite our work:
如果我们的工作对你有所启发或你在研究中使用了我们的代码库,请引用我们的工作:

@inproceedings{dai2023disentangling,
  title={Disentangling Writer and Character Styles for Handwriting Generation},
  author={Dai, Gang and Zhang, Yifan and Wang, Qingfeng and Du, Qing and Yu, Zhuliang and Liu, Zhuoman and Huang, Shuangping},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
  pages={5977--5986},
  year={2023}
}

⭐ StarGraph

Star History Chart

About

This repository is the official implementation of Disentangling Writer and Character Styles for Handwriting Generation (CVPR 2023)
此仓库是 Disentangling Writer and Character Styles for Handwriting Generation (CVPR 2023) 的官方实现

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages