How we write rebuttals 我们如何写反驳
By Devi Parikh, Dhruv Batra, Stefan Lee
作者：Devi Parikh、Dhruv Batra、Stefan Lee
We frequently find ourselves giving the same advice to different students on how to write rebuttals. So we thought we’d write it up. Our experience is with AI conferences (e.g., CVPR, ECCV, ICCV, NeurIPS, ICLR, EMNLP).The core guiding principle is that the rebuttal should be thorough, direct, and easy for the Reviewers and Area Chair (RACs) to follow.
我们经常发现自己就如何撰写反驳向不同的学生提供相同的建议。所以我们认为我们会把它写下来。我们的经验来自 AI 会议（例如，CVPR、ECCV、ICCV、NeurIPS、ICLR、EMNLP）。核心指导原则是反驳应该彻底、直接，并且易于评审员和领域主席 (RAC) 遵循。
Why do we write rebuttals?
我们为什么要写反驳？
In a phrase — to clarify and convince. Science is a deliberative process, and rebuttals are simply a stage in that process.
一句话——澄清和说服。科学是一个审议过程，反驳只是该过程中的一个阶段。
Bill Freeman gave an excellent talk at the “Good Citizen of CVPR” panel at CVPR18. He talked about the idealized image of the peer review process that new members of a research community often hold — they imagine scholars, with plenty of time on their hands, gathered around a quiet space carefully pouring over the nuances of their work.
Bill Freeman 在 CVPR18 的“CVPR 好公民”小组中发表了精彩演讲。他谈到了研究社区的新成员经常持有的同行评审过程的理想化形象——他们想象学者们手头有充足的时间，聚集在一个安静的空间周围，仔细地倾诉他们工作的细微差别。
The reality, Bill describes, is more like a large crowded marketplace. It’s messy and it’s chaotic; different actors in the process are trying to achieve different goals, each operating under limited time and attention-spans, partial information, while juggling multiple responsibilities.
比尔描述说，现实更像是一个拥挤的大市场。它很乱，很混乱；这个过程中的不同参与者正在努力实现不同的目标，每个人都在有限的时间和注意力、部分信息下运作，同时兼顾多重责任。
With this in mind, it is important to note who we write rebuttals for and what our goal is.
考虑到这一点，重要的是要注意我们为谁写反驳以及我们的目标是什么。
We write rebuttals for two different audiences
我们为两种不同的听众写反驳
- The reviewers, who have read your paper (to varying degrees), but may have forgotten some of the details or didn’t understand them in the first place.
审稿人，他们已经阅读了你的论文（不同程度），但可能已经忘记了一些细节或者一开始就不理解它们。 - The AC, who is likely even less familiar with your work, and a good guiding principle is to assume that all they will read is the set of reviews and the rebuttal.
AC 可能更不熟悉你的工作，一个好的指导原则是假设他们将阅读的只是一组评论和反驳。
Our goals are 我们的目标是
- For the reviewers: clarify doubts, answer questions, correct misunderstandings, push back on mischaracterizations, and make a good-faith effort to incorporate feedback and improve your work.
对于审稿人：澄清疑问、回答问题、纠正误解、反驳错误的描述，并真诚地努力吸收反馈并改进你的工作。 - For the AC: convince them that you have made a good-faith effort, present a representative summary of the reviews, help them understand if the reviewer concerns were addressed, call out bad-faith reviewing, and ultimately, help them make a decision.
对于 AC：让他们相信您做出了善意的努力，提供有代表性的评论摘要，帮助他们了解评论者的担忧是否得到解决，呼吁恶意评论，并最终帮助他们做出决定。
In our experience, most new members of the research community focus on (1), but ignore (2).
根据我们的经验，研究界的大多数新成员都关注 (1)，而忽略 (2)。
An imperfect but useful metaphor for rebuttals is debate competitions. Yes, we are trying to convince our opponent (and this is where the metaphor is imperfect; reviewers are not our opponents, but hang with us). But more importantly, we are trying to convince the judges, who will ultimately be making the decisions. Thus, all else being equal, it is more important to convince the judges of your arguments than change your opponents’ minds.
一个不完美但有用的反驳比喻是辩论赛。是的，我们正在努力说服我们的对手（这就是比喻不完美的地方；审稿人不是我们的对手，而是支持我们）。但更重要的是，我们正在努力说服最终将做出决定的法官。因此，在其他条件相同的情况下，让法官相信你的论点比改变对手的想法更重要。
We provide concrete recommendations below, but a good short-hand is the following — would a neutral third-party be able to tell if the reviewer concerns were addressed purely based on your rebuttal (without reading the paper or the reviews again)?
我们在下面提供了具体的建议，但下面是一个很好的速记——中立的第三方是否能够判断审稿人的担忧是否纯粹基于你的反驳（无需再次阅读论文或评论）？
Our Process. 我们的流程。
- Itemize reviewer comments. We use a handy spreadsheet to organize individual comments/questions/concerns raised by each reviewer. Laying everything out in the same place helps identify common concerns and keeps us from missing anything accidentally. Do this ASAP to identify any necessary new experiments (if the venue allows it) or analysis early.
逐条列出审稿人的意见。我们使用一个方便的电子表格来组织每个评论者提出的个人评论/问题/疑虑。将所有内容放在同一个地方有助于确定共同关注的问题，并防止我们意外遗漏任何内容。尽快执行此操作以识别任何必要的新实验（如果场地允许）或尽早进行分析。 - Brain dump possible responses. The sheet has a column for each author to reply to each reviewer comment. Dump your thoughts here in rough text without regard for style or length. Being convincing and concise is a subtractive process.
大脑转储可能的反应。该表有一个列供每个作者回复每个审稿人的评论。在这里用粗略的文字表达你的想法，不考虑风格或长度。令人信服和简洁是一个减法过程。 - Write a draft rebuttal. Turn your consensus in the sheet into concrete responses in a rebuttal draft. Write concisely but don’t worry about space. Cover every point and trim / prioritize them later.
写反驳草稿。将您在表格中的共识转化为反驳草案中的具体回应。写得简洁，但不要担心空间。涵盖每一点并在以后修剪/优先考虑它们。 - Review and revise. Reread the initial reviews and your sheet to make sure everything has been addressed. Prioritize major concerns and start working towards meeting space limitations.
审查和修改。重新阅读最初的评论和您的表格，以确保所有内容都已得到解决。优先考虑主要问题并开始努力满足空间限制。
Example set of initial responses from authors is shown below. You’ll see more polished snippets of some of these responses through the rest of this post
作者的初始回复示例集如下所示。在本文的其余部分，您将看到其中一些回复的更精美的片段
As for putting together the rebuttal itself, below is a list of tips and principles we follow and believe are effective. Others likely have their own.
至于将反驳本身放在一起，下面列出了我们遵循并认为有效的提示和原则。其他人可能有自己的。
Tip 1. Start positive. Provide a summary of the reviews, highlighting positive things that reviewers said about your work. Rebuttals focus mostly on responding to perceived negatives, don’t let RACs forget about the strengths along the way.
提示 1. 从积极开始。提供评论摘要，突出评论者对您的工作所说的正面评价。反驳主要集中在回应感知到的负面因素上，不要让 RAC 忘记沿途的优势。
Tip 2. Order matters. Start with the biggest concerns that you have good answers for and work your way to less clear-cut responses and minor points.
提示 2. 顺序很重要。从你有好的答案的最大问题开始，然后逐步解决不太明确的回应和次要问题。
Tip 3. Let reviewers speak for themselves, then respond directly. Quote the core of the reviewer’s question or concern concisely and completely. Then before saying anything else, respond to it directly. And then give details, describe context, or explain your position.
提示 3. 让审稿人自己发言，然后直接回复。简明完整地引用审稿人问题或疑虑的核心内容。然后在说任何其他话之前，直接回应它。然后提供详细信息、描述上下文或解释您的立场。
- “Are these averaged across multiple runs?”:
“这些是多次运行的平均值吗？”：
“Yes, we averaged across 5 random seeds.”
“是的，我们对 5 个随机种子进行了平均。” - “Are the segmentation masks used during training?”:
“训练期间是否使用分割掩码？”：
“No, they are only used to evaluate our results.”
“不，它们只是用来评估我们的结果。” - “So overall, the proposed approach needs more human annotations than the baseline.”:
“总的来说，所提出的方法比基线需要更多的人工注释。”：
“Not quite. While the first few iterations of our approach need a human in the loop more often, as we collect more data, our approach relies on humans less than the baseline. For typical dataset sizes, we use fewer annotations overall. E.g., for GNART20 with 10k images, our approach uses 2.4k human annotations while the baseline uses 2.8k”
“不完全的。虽然我们方法的前几次迭代更频繁地需要人参与循环，但随着我们收集更多数据，我们的方法对人的依赖程度低于基线。对于典型的数据集大小，我们总体上使用较少的注释。例如，对于具有 10k 图像的 GNART20，我们的方法使用 2.4k 人工注释，而基线使用 2.8k” - “Does your baseline match the one reported by Smith et al. at CVPR last year [43]?”:
“您的基线是否与 Smith 等人报告的基线相符？在去年的 CVPR [43]？”：
“Almost. The is a 0.1% difference in performance between [43]’s publicly available code and what is reported in [43]. Our baseline matches the former.”
“几乎。 [43] 的公开可用代码与 [43] 中报告的代码之间的性能差异为 0.1%。我们的基线与前者相符。” - “Did you evaluate on realistic environments?”:
“您是否在现实环境中进行了评估？”：
“We disagree with the question’s premise. While these environments are simulated, they are highly photorealistic.”
“我们不同意这个问题的前提。虽然这些环境是模拟的，但它们非常逼真。” - “Why did you not compare to GMAP?“
“你为什么不与 GMAP 进行比较？”
“GMAP is prohibitively expensive in our setting. Our environments have a significantly larger state-space (10k) than BRIE (20) which has been used to evaluate GMAP thus far. Back-of-the-envelope calculations suggest it would take 128 GPUS for 3 months to evaluate GMAP.”
“GMAP 在我们的环境中非常昂贵。我们的环境具有比迄今为止用于评估 GMAP 的 BRIE (20) 大得多的状态空间 (10k)。粗略计算表明，评估 GMAP 需要 128 个 GPUS 3 个月。”
Tip 4. Be conversational. Notice the conversational nature of the example responses above. It makes it easier for RACs to follow, and the responses are less likely to be perceived as being combative.
提示 4. 保持对话。请注意上面示例响应的对话性质。它使 RAC 更容易遵循，并且响应不太可能被视为好斗。
Tip 5. Respond to the intent of the questions. Don’t feel trapped to only discuss the quoted concern — also address the intent of the comment. For example, “Why didn’t you evaluate on GLORP3?” may generally be calling your experiments into question. Answer, but then point out that you’ve already evaluated on X,Y, and Z which should be sufficient! Note that it is useful for other RACs to be reminded of your extensive experimental evaluation. A first glance at a reviewer comment suggesting otherwise could leave a false impression.
提示 5. 回答问题的意图。不要觉得只讨论引用的问题——还要解决评论的意图。例如，“你为什么不在 GLORP3 上进行评估？”通常可能会质疑您的实验。回答，然后指出您已经对 X、Y 和 Z 进行了评估，这应该足够了！请注意，提醒其他 RAC 进行广泛的实验评估很有用。乍一看审稿人的评论可能会留下错误的印象。
Tip 6. Don’t be afraid of emphasis. “Row 2 in Table 4 shows exactly that.” “We do NOT need a human-in-the-loop at test time.” Notice that many of the responses above are not just direct, but also have emphasis (in tone if not formatting of text).
提示 6. 不要害怕强调。 “表 4 中的第 2 行显示了这一点。” “我们在测试时不需要人在环路中。”请注意，上面的许多响应不仅是直接的，而且还具有强调（如果不是文本格式，则在语气中）。
Tip 7. Feel free to set the stage. If it seems like all reviewers missed a central point, a concise, crisp recap of the main point could help.
提示 7. 随意布置舞台。如果所有评论者似乎都错过了一个中心点，那么对要点进行简明扼要的回顾可能会有所帮助。
Tip 8. Keep things self-contained. Assume RACs don’t remember much about your paper and that they likely won’t read it again in detail. Re-introduce any acronyms, remind them of relevant details of an experimental setup. Notice that all the responses above likely make sense to you even though you may be unfamiliar with the papers (and in some cases the names and details are made up).
提示 8. 保持独立。假设 RAC 不太记得你的论文，他们可能不会再详细阅读它。重新引入任何首字母缩略词，提醒他们实验设置的相关细节。请注意，即使您可能不熟悉这些论文（并且在某些情况下，名称和详细信息是编造的），上面的所有回答对您来说都可能有意义。
Tip 9. But get credit for details you already included. That said, if something a reviewer asked for was already in the paper, say so. Give them line/Table/Figure numbers, and then restate it in the rebuttal. The references back to the main paper are to establish credibility with all RACs that the paper was not lacking important details. (They are not necessarily to have RACs go back and look at the paper.)
提示 9. 但要为你已经包含的细节赢得赞誉。也就是说，如果审稿人要求的内容已经在论文中，请直接说出来。给他们行/表/图编号，然后在反驳中重申。对主要文件的引用是为了建立所有 RAC 的可信度，即该文件并不缺少重要的细节。 （他们不一定要让 RAC 回去看论文。）
[R1: baseline that retrieves images from caption]
[R1：从标题中检索图像的基线]
As described in L341, 746, 772, round 0 is the caption. Thus the round 0 point in Fig 4a is the caption-based image retrieval baseline. At the best-performing round, RL-Full-QAf achieves ~3% percentile (~285 absolute rank) improvement over RL-Full-QAf round 0.
如 L341、746、772 中所述，第 0 轮是标题。因此，图 4a 中的第 0 轮点是基于字幕的图像检索基线。在表现最好的一轮中，RL-Full-QAf 比第 0 轮 RL-Full-QAf 实现了约 3% 的百分位数（约 285 绝对等级）改进。
Tip 10. Consolidate common concerns. Save space by responding to multiple reviewers at once if they share related concerns.
技巧 10. 巩固共同关注的问题。如果他们有共同的关注点，可以通过一次回复多个审阅者来节省空间。
Tip 11. Color-code reviewers. Notice above the trick to color-code reviewers. Make it as easy as possible for reviewers to spot responses that are relevant to them — even when things are merged or not in reviewer order.
技巧 11. 颜色代码审阅者。请注意上面的颜色代码审阅者的技巧。让审稿人尽可能容易地发现与他们相关的回复——即使事情被合并或不符合审稿人的顺序。
Tip 12. Stats speak louder than words. Rather than argue with RACs, give them data/stats to back your claim up. These can be statistics/analysis of your data or results. Or the results of additional experiments you run to respond to their concern (if allowed by the venue). Every time you find yourself having a different opinion than the reviewer, ask if you can establish that with data. You can always provide the intuitive arguments after you’ve settled the issue with data.
提示 12. 数据胜于雄辩。与其与 RAC 争论，不如向他们提供数据/统计数据来支持您的主张。这些可以是您的数据或结果的统计/分析。或者你为回应他们的担忧而进行的额外实验的结果（如果场地允许）。每当你发现自己与审稿人有不同意见时，问问你是否可以用数据证明这一点。在用数据解决问题后，你总是可以提供直观的论据。
Tip 13. Don’t promise, do. Instead of saying “We will discuss Singh et al. in the paper.”, provide a discussion in the rebuttal. Instead of saying “We will explain what D_{RT} stands for in the paper”, explain what it stands for in the rebuttal. And then also add that you will add it to the paper. It makes it significantly easier for RACs to trust that you will make the promised changes.
提示 13. 不要承诺，做。而不是说“我们将讨论 Singh 等人。在论文中。”，在反驳中提供讨论。不要说“我们将解释 D_{RT} 在论文中代表什么”，而是在反驳中解释它代表什么。然后还要补充说，您会将其添加到论文中。它使 RAC 更容易相信您会做出承诺的更改。
Tip 14. Be receptive and reasonable. Most RACs will appreciate it. Plus, it is just the better thing to do — these are your colleagues! :)
提示 14. 善于接受和通情达理。大多数 RAC 都会对此表示赞赏。另外，这样做是更好的选择——这些是您的同事！ :)
Likewise, instead of arguing why a reviewer’s suggestion might not work, try it and see if it works. Maybe it will!
同样，与其争论为什么审稿人的建议可能行不通，不如尝试看看是否行得通。也许会！
Tip 15. Be transparent. Reviewers hinted at an additional experiment but the venue doesn’t allow it? Say so. They asked about intuitions about a trend but you don’t have any? Say that you’ve thought about it but don’t have any good ideas, and will continue to investigate it. Don’t have enough GPUs to run the experiment they asked for? Say so.
提示 15. 保持透明。审稿人暗示要进行额外的实验，但场地不允许？这样说的。他们问了关于趋势的直觉，但你没有？说你想过，但没有什么好主意，会继续研究。没有足够的 GPU 来运行他们要求的实验？这样说的。
Tip 16. Shine a spotlight on reviewers acting in bad-faith. In some circumstances, a reviewer may not be adhering to reviewing best practices or may not have taken the reviewing role seriously. It can be important to make sure the other RACs realize this and appropriately discount their review. Pointing out unreasonable or unsubstantiated comments and referencing other reviewers that disagree can help. This can also include confidential comments to the AC (where applicable).
提示 16. 关注审稿人的恶意行为。在某些情况下，审阅者可能没有遵守审阅最佳实践，或者可能没有认真对待审阅角色。确保其他 RAC 意识到这一点并适当地低估他们的审查可能很重要。指出不合理或未经证实的评论并引用其他不同意的审稿人会有所帮助。这还可以包括对 AC 的机密评论（如适用）。
Tip 17. Acknowledge reviewer efforts. On the other hand, if a reviewer goes above and beyond to be constructive, thank them for it. Typo list? Thank you. Pointers to relevant work? Thank you. Detailed musings about future work? Thank you. Add at least a short blurb acknowledging these things!
提示 17. 承认审稿人的努力。另一方面，如果审稿人超越了建设性，感谢他们。打字清单？谢谢。相关工作的指针？谢谢。关于未来工作的详细思考？谢谢。至少添加一个简短的宣传语来承认这些事情！
Tip 18. Don’t forget the humans on the other end. Keep in mind that this is not just a scientific but a sociopolitical interaction with other humans :) So decide whether you’d like to be argumentative and risk your reviewer taking a strong stand against the paper, or if you’d like to work towards a common ground. Finding points where you do agree with the reviewer and acknowledging them can help with the latter.
提示 18. 不要忘记另一端的人类。请记住，这不仅是一种科学，而且是一种与其他人的社会政治互动 :) 因此，请决定你是愿意争论不休并冒着你的审稿人对论文采取强硬立场的风险，还是你愿意朝着这个方向努力一个共同点。找到你同意审稿人的观点并承认他们可以帮助后者。
We hope this was helpful, and happy rebutting :)
我们希望这对您有所帮助，并乐于反驳 :)
