ReaRAG: Knowledge-guided Reasoning Enhances Factuality of Large Reasoning Models with Iterative Retrieval Augmented Generation
ReaRAG：通过迭代检索增强生成增强大型推理模型的知识引导推理的事实性

Large Reasoning Models (LRMs) such as OpenAI’s o1 (Jaech et al., 2024), Qwen’s QwQ32B ${}^1$${}^1$^(1){ }^{1} (Team, 2024), GLM-Z1 ${}^2$${}^2$^(2){ }^{2} and DeepSeek-R1 (DeepSeek-AI et al., 2025) demonstrate impressive reasoning capabilities on complex tasks (Xu et al., 2025). However, their reliance on parametric knowledge during reasoning limits performance
像 OpenAI 的 o1（Jaech 等，2024）、阿里的 QwQ32B ${}^1$${}^1$^(1){ }^{1} （Team，2024）、GLM-Z1 ${}^2$${}^2$^(2){ }^{2} 和 DeepSeek-R1（DeepSeek-AI 等，2025）这样的大规模推理模型在复杂任务中展现出令人印象深刻的推理能力。然而，它们在推理过程中对参数化知识的依赖限制了性能

Figure 1: Unlike LRMs, ReaRAG iteratively constructs knowledge-guided reasoning chains for factual answers.
图 1：与大规模推理模型不同，ReaRAG 迭代构建知识引导的推理链以获取事实性答案。
on question answering (QA) tasks that require factual answers, where reasoning beyond the model’s parametric knowledge is required.
在需要事实性答案的问答（QA）任务中，需要超越模型参数知识的推理能力。

To enhance LRMs’ factuality, RetrievalAugmented Generation (RAG) (Lewis et al., 2020; Shi et al., 2024; Guu et al., 2020) offers a promising solution by integrating external knowledge but faces challenges in retrieving relevant documents, which requires formulating precise search queries (Chan et al., 2024). Prior research has explored iterative retrieval strategies (Press et al., 2023; Shao et al., 2023), which construct reasoning chains of sub-queries and sub-answers to solve multi-hop QA. However, these methods suffer from error propagation, where mistakes in earlier steps mislead subsequent retrieval and reasoning, ultimately degrading the overall answer quality (Cao et al., 2023).
为了增强大型检索模型（LRMs）的事实性，检索增强生成（RAG）（Lewis 等，2020；Shi 等，2024；Guu 等，2020）通过整合外部知识提供了一个有希望的解决方案，但在检索相关文档时面临挑战，这需要制定精确的搜索查询（Chan 等，2024）。先前的研究探索了迭代检索策略（Press 等，2023；Shao 等，2023），这些策略构建子查询和子答案的推理链以解决多跳问答。然而，这些方法受到错误传播的困扰，其中早期步骤的错误误导后续检索和推理，最终降低了整体答案质量（Cao 等，2023）。

To address this limitation, Search-o1 (Li et al., 2025) prompts a LRM to iteratively retrieve documents and generate sub-answers via a Reason-inDocuments module, relying heavily on the model’s inherent reasoning and instruction-following capabilities. This leads to three key issues: (1) unreliable retrieval token generation, (2) information extraction failures and hallucinations during document reasoning, and (3) overthinking in multi-hop QA (Chen et al., 2024). Effectively integrating reasoning with retrieval remains challenging.
为了解决这一局限性，Search-o1（Li 等，2025）提示大型检索模型通过文档内推理模块迭代地检索文档并生成子答案，严重依赖模型固有的推理和指令遵循能力。这导致了三个关键问题：（1）不可靠的检索标记生成，（2）文档推理中的信息提取失败和幻觉，（3）多跳问答中的过度思考（Chen 等，2024）。有效地将推理与检索相结合仍然具有挑战性。

In response to this challenge, recent works such as R1-Searcher (Song et al., 2025) and Re-
针对这一挑战，最近的工作如 R1-Searcher（Song 等，2025）和 Re-

Search (Chen et al., 2025) adopt reinforcement learning (RL) to incentivize Large Language Models (LLMs) to perform reasoning before retrieval. However, such policy optimization approaches involve complex training paradigms and require extensive computational resources.
陈等人（Chen et al., 2025）采用强化学习（RL）来激励大型语言模型（LLMs）在检索前执行推理。然而，这种策略优化方法涉及复杂的训练范式，并需要大量计算资源。

In this paper, we propose $\mathbf{R}\mathbf{e}\mathbf{a}\mathbf{R}\mathbf{A}\mathbf{G}$$\mathbf{R}\mathbf{e}\mathbf{a}\mathbf{R}\mathbf{A}\mathbf{G}$ReaRAG\mathbf{R e a R A G}, a factualityenhanced reasoning model for RAG that achieves comparable performance without relying on complex RL-based training. Instead of resource-heavy policy optimization, we strategically distill the reasoning capabilities of LRMs to iteratively construct knowledge-guided reasoning chains, which are then curated into a dedicated dataset with restricted chain lengths to fine-tune ReaRAG under the Thought-Action-Observation paradigm. During inference, ReaRAG iteratively performs the search action and dynamically decides when to trigger the finish action, avoiding excessive retrieval. Guided by external knowledge, ReaRAG reflects on its reasoning trajectory, detects errors, and realigns its reasoning toward the correct path, leading to improved performance with greater training efficiency and practical implementation.
在本文中，我们提出 $\mathbf{R}\mathbf{e}\mathbf{a}\mathbf{R}\mathbf{A}\mathbf{G}$$\mathbf{R}\mathbf{e}\mathbf{a}\mathbf{R}\mathbf{A}\mathbf{G}$ReaRAG\mathbf{R e a R A G} ，这是一个事实性增强的检索增强生成（RAG）推理模型，通过不依赖复杂的基于强化学习的训练来实现可比较的性能。我们不是使用资源密集型策略优化，而是战略性地提取大型检索模型（LRMs）的推理能力，迭代构建知识引导的推理链，并将其整理成一个具有受限链长度的专用数据集，以在思考-行动-观察范式下微调 ReaRAG。在推理过程中，ReaRAG 迭代执行搜索动作并动态决定何时触发完成动作，避免过度检索。在外部知识的指导下，ReaRAG 反思其推理轨迹，检测错误，并将推理重新对准正确路径，从而以更高的训练效率和实际可实施性带来性能提升。

To validate our approach, we conduct experiments on six benchmarks covering both multi-hop and single-hop QA, where ReaRAG achieves the highest overall performance. In summary, our contributions are as follows:
为验证我们的方法，我们在六个涵盖多跳和单跳问答的基准测试上进行实验，ReaRAG 在这些测试中取得了最高的整体性能。总的来说，我们的贡献如下：

Reasoning-enhanced LLMs. Numerous works have investigated how to elicit the reasoning abilities of LLMs. Early approaches, such as Chain-of-Thought (COT) (Wei et al., 2022), ReAct (Yao et al., 2023b) and Tree of Thought (ToT) (Yao et al., 2023a) prompts LLMs to generate human-like step by step reasoning chain, though they still struggle on complex reasoning tasks. Recent advances of LRMs have scaled up CoT through RL (Kumar et al., 2024), enabling models to generate long CoT before providing a final answer. Notably, LRMs include OpenAI’s o1 (Jaech et al., 2024), Qwen’s QwQ-32B (Team, 2024), GLM-Z1 and DeepSeek-R1(DeepSeek-AI et al., 2025) demonstrate impressive performance in complex reasoning tasks. Despite these advancements, most LRMs lack the ability to interact with external knowledge, limiting their capacity to generate factual responses.
基于推理增强的大语言模型（LLMs）。众多研究已探索如何激发大语言模型的推理能力。早期方法，如思维链（Chain-of-Thought，COT）（Wei 等，2022）、ReAct（Yao 等，2023b）和思维树（Tree of Thought，ToT）（Yao 等，2023a）提示大语言模型生成类似人类的逐步推理链，尽管它们在复杂推理任务上仍面临困难。最近，大语言模型的进展通过强化学习（RL）扩展了思维链（Kumar 等，2024），使模型能在给出最终答案之前生成长的思维链。值得注意的是，大语言模型包括 OpenAI 的 o1（Jaech 等，2024）、Qwen 的 QwQ-32B（Team，2024）、GLM-Z1 和 DeepSeek-R1（DeepSeek-AI 等，2025）在复杂推理任务中展现了令人印象深刻的性能。尽管取得了这些进展，但大多数大语言模型仍缺乏与外部知识交互的能力，限制了其生成事实性响应的能力。

Retrieval-Augmented Generation. RAG has emerged as a promising paradigm for improving LLMs’ factuality. Early methods rely on a single retrieval step (Lewis et al., 2020; Borgeaud et al., 2022; Izacard et al., 2023), which often falls short for multi-hop QA tasks due to limited retrieval quality. To mitigate noisy retrieval (Shi et al., 2023), Self-RAG (Asai et al., 2024) and CRAG (Yan et al., 2024) introduce reflection mechanisms on the retrieved documents, yet single-step retrieval remains insufficient. To address these limitations, iterative methods such as Iter-RetGen (Shao et al., 2023) and Self-Ask (Press et al., 2023) progressively retrieve relevant documents to gather sufficient information for final answer. Building on this, SearChain (Xu et al., 2024) first generates a complete reasoning chain, then verifies the answer of each node in the chain based on external knowledge. However, these approaches lack a strong reflection mechanism to recover from earlier reasoning errors.
检索增强生成（RAG）。检索增强生成已成为提升大语言模型事实性的一个有前景的范式。早期方法依赖单一检索步骤（Lewis 等，2020；Borgeaud 等，2022；Izacard 等，2023），这通常无法满足多跳问答任务，因为检索质量有限。为缓解噪声检索（Shi 等，2023），Self-RAG（Asai 等，2024）和 CRAG（Yan 等，2024）在检索文档上引入反思机制，但单步检索仍然不够充分。为解决这些限制，迭代方法如 Iter-RetGen（Shao 等，2023）和 Self-Ask（Press 等，2023）逐步检索相关文档，以收集足够信息得出最终答案。在此基础上，SearChain（Xu 等，2024）首先生成完整的推理链，然后基于外部知识验证链中每个节点的答案。然而，这些方法缺乏强大的反思机制来从早期推理错误中恢复。

Reasoning-enhanced RAG. Recent studies in scaling token generation at test-time (OpenAI, 2024; Muennighoff et al., 2025; Snell et al., 2024) to enhance LLMs’ reasoning capabilities have spurred interest in reasoning-enhanced RAG. RAGStar (Jiang et al., 2024) leverages Monte Carlo Tree Search to iteratively decompose multi-hop ques-
推理增强的检索增强生成（RAG）。最近关于测试时标记生成扩展的研究（OpenAI，2024；Muennighoff 等，2025；Snell 等，2024）旨在增强 LLMs 的推理能力，这引发了对推理增强 RAG 的兴趣。RAGStar（Jiang 等，2024）利用蒙特卡洛树搜索通过奖励模型迭代分解多跳问题

Figure 2: Overview of our approach to develop a factuality-enhanced reasoning model RearAG. To equip ReaRAG with knowledge-guided reasoning ability, we propose an automated data construction approach (Algorithm 1). Next, we fine-tune ReaRAG on the constructed dataset to conduct reasoning iteratively, following the Thought-ActionObservation Paradigm to solve complex queries. Pseudocode for the inference stage is provided in Algorithm 2.
图 2：我们开发事实性增强推理模型 ReaRAG 的方法概述。为了为 ReaRAG 配备知识引导的推理能力，我们提出了一种自动化数据构建方法（算法 1）。接下来，我们在构建的数据集上对 ReaRAG 进行微调，遵循思考-行动-观察范式迭代推理以解决复杂查询。推理阶段的伪代码在算法 2 中提供。
tions, guided by a reward model during the tree expansion. Search-o1 (Li et al., 2025) introduces a document reasoning module and prompts QwQ32B to generate special tokens that trigger knowledge retrieval. However, it heavily depends on the base model’s instruction-following and inherited reasoning capabilities, leading to three key challenges: unreliable generation of specialized tokens, failure in extracting information from retrieved documents, and overthinking for multi-hop QA. To integrate reasoning with retrieval, R1-Searcher (Song et al., 2025) and ReSearch (Chen et al., 2025) adopt a RL-based method that incentivizes LLMs to reason before retrieving. However, despite the complexity of RL-based training paradigms, our approach achieves comparable performance through strategic distillation, offering greater training efficiency and practical ease of implementation.
题。搜索-o1（Li 等，2025）引入了文档推理模块，并提示 QwQ32B 生成触发知识检索的特殊标记。然而，它高度依赖基础模型的指令遵循和继承的推理能力，这导致三个关键挑战：专门标记生成不可靠、从检索到的文档中提取信息失败，以及多跳问题问答过度思考。为了将推理与检索整合，R1-Searcher（Song 等，2025）和 ReSearch（Chen 等，2025）采用基于强化学习的方法，激励 LLMs 在检索前进行推理。然而，尽管强化学习训练范式复杂，我们的方法通过策略蒸馏实现了可比的性能，提供了更高的训练效率和更实际的实施便利性。

In this section, we first formalize the task, then present our novel approach for developing ReaRAG as illustrated in Figure 2.
在本节中，我们首先正式化任务，然后如图 2 所示，介绍我们开发 ReaRAG 的新颖方法。

We focus on the multi-hop QA task, where iterative reasoning improves answer accuracy. Given a
我们关注多跳问答任务，通过迭代推理提高答案准确性。给定一个
question $x$$x$xx, our goal is to construct a knowledgeguided reasoning chain $\mathcal{C}$$\mathcal{C}$C\mathcal{C} to enhance the factual correctness of the generated answer $\hat{y}$$\widehat{y}$hat(y)\hat{y}. Specifically, the reasoning chain is formulated as a sequence of $N$$N$NN steps, where each step consists of a reasoning thought ${\tau }_t$${\tau }_t$tau_(t)\tau_{t}, an action ${\alpha }_t$${\alpha }_t$alpha_(t)\alpha_{t}, and an observation $o_t$$o_t$o_(t)o_{t} :
问题 $x$$x$xx ，我们的目标是构建一个知识引导的推理链 $\mathcal{C}$$\mathcal{C}$C\mathcal{C} ，以增强生成答案 $\hat{y}$$\widehat{y}$hat(y)\hat{y} 的事实准确性。具体而言，推理链被表述为 $N$$N$NN 步骤的序列，其中每一步包括一个推理思路 ${\tau }_t$${\tau }_t$tau_(t)\tau_{t} 、一个动作 ${\alpha }_t$${\alpha }_t$alpha_(t)\alpha_{t} 和一个观察结果 $o_t$$o_t$o_(t)o_{t} ：

The number of reasoning steps $N$$N$NN is dynamically determined by the model but is constrained by an upper limit $T_{\text{max}}$$T_{\text{max }}$T_("max ")T_{\text {max }} to prevent indefinite iterations, i.e., $N\le T_{max}$$N\le T_{max}$N <= T_(max)N \leq T_{\max }. To guide the reasoning process with external knowledge, we define the action space as $\mathcal{A}=\{\mathrm{search}()$$\mathcal{A}=\{\mathrm{search}()$A={search()\mathcal{A}=\{\operatorname{search}(), finish() $\}$$\}$}\} where search takes a search query as input, while the input to finish represents the derived answer. At each step $t$$t$tt, the model refines a search query based on the reasoning thought ${\tau }_t$${\tau }_t$tau_(t)\tau_{t} and executes the search action to retrieve relevant information from the RAG engine $\mathcal{R}$$\mathcal{R}$R\mathcal{R}. The process continues until the model selects the finish action, at which point the final answer $\hat{y}$$\widehat{y}$hat(y)\hat{y} is derived from all prior reasoning steps. This ensures that the answer is grounded in retrieved knowledge through an iterative and structured reasoning process, thereby enhancing the factual reliability of LRMs.
推理步骤的数量 $N$$N$NN 由模型动态确定，但受上限 $T_{\text{max}}$$T_{\text{max }}$T_("max ")T_{\text {max }} 约束，以防止无限迭代，即 $N\le T_{max}$$N\le T_{max}$N <= T_(max)N \leq T_{\max } 。为了利用外部知识指导推理过程，我们将动作空间定义为 $\mathcal{A}=\{\mathrm{search}()$$\mathcal{A}=\{\mathrm{search}()$A={search()\mathcal{A}=\{\operatorname{search}() ，finish() $\}$$\}$}\} 其中搜索操作接受搜索查询作为输入，而 finish 的输入表示推导出的答案。在每一步 $t$$t$tt ，模型根据推理思路 ${\tau }_t$${\tau }_t$tau_(t)\tau_{t} 优化搜索查询，并执行搜索动作从 RAG 引擎 $\mathcal{R}$$\mathcal{R}$R\mathcal{R} 检索相关信息。该过程持续进行，直到模型选择 finish 动作，此时最终答案 $\hat{y}$$\widehat{y}$hat(y)\hat{y} 由之前的所有推理步骤推导而出。这确保了答案通过迭代和结构化的推理过程扎根于检索到的知识，从而提高大规模语言模型的事实可靠性。

Algorithm 1 Data Construction
Input: Seed Dataset \(\mathcal{D}_{\text {seed }}\), Large Reasoning
Model \(\mathcal{M}_{L R M}\), Instruction prompt \(\mathcal{P}_{d}\), Max itera-
tions \(T_{\text {max }}\), RAG engine \(\mathcal{R}\)
Output: Dataset \(\mathcal{D}_{\text {reason }}\) with reasoning chains
    Initialize \(\mathcal{D}_{\text {reason }} \leftarrow \emptyset\)
    for each \(\left(x_{i}, d o c_{i}\right) \sim \mathcal{D}_{\text {seed }}\) do
            \(\triangleright\) Sample question and gold documents
        \(t \leftarrow 0 \quad \triangleright\) Iteration counter
        \(\mathcal{C}_{i} \leftarrow[] \quad \triangleright\) Reasoning chain
        while \(t<T_{\text {max }}\) do
            \(y_{t}^{\prime} \leftarrow \mathcal{M}_{L R M}\left(\left[\mathcal{P}_{d} \oplus x_{i} \oplus \mathcal{C}_{i}\right]\right)\)
                    \(\triangleright\) Generate response
            \(\left(\tau_{t}, \alpha_{t}\right) \leftarrow \operatorname{parse}\left(y_{t}^{\prime}\right)\)
                \(\triangleright\) Extract thought \(\tau_{t}\) and action \(\alpha_{t}\)
            Get action type \(\alpha_{t_{\text {type }}} \leftarrow \alpha_{t}\) ['type']
            if finish \(\in \alpha_{t_{\text {type }}}\) then
                Append \(\left(\tau_{t}, \alpha_{t}\right)\) to \(\mathcal{C}_{i}\)
                break
            else if search \(\in \alpha_{t_{\text {type }}}\) then
                \(q_{s} \leftarrow \alpha_{t}\) ['query']
                \(o_{t} \leftarrow \mathcal{R}\left(q_{s}, d o c_{i}\right)\)
                    \(\triangleright\) Get \(o_{t}\) from RAG engine
                Append \(\left(\tau_{t}, \alpha_{t}, o_{t}\right)\) to \(\mathcal{C}_{i}\)
            end if
            \(t \leftarrow t+1\)
        end while
        Append \(\mathcal{C}_{i}\) to \(\mathcal{D}_{\text {reason }}\)
    end for
    return \(\mathcal{D}_{\text {reason }}\)

While existing LRMs demonstrate strong reasoning capabilities, they often fail to ground their reasoning process in factual knowledge. To make external knowledge accessible, we design a structured reasoning step where each step consists of a reasoning thought ${\tau }_t$${\tau }_t$tau_(t)\tau_{t}, an action ${\alpha }_t$${\alpha }_t$alpha_(t)\alpha_{t}, and an observation $o_t$$o_t$o_(t)o_{t}.
尽管现有的大规模语言模型展现出强大的推理能力，但它们常常未能将推理过程建立在事实知识的基础上。为了使外部知识可访问，我们设计了一个结构化的推理步骤，每一步由推理思路 ${\tau }_t$${\tau }_t$tau_(t)\tau_{t} 、动作 ${\alpha }_t$${\alpha }_t$alpha_(t)\alpha_{t} 和观察结果 $o_t$$o_t$o_(t)o_{t} 组成。

To equip ReaRAG with the ability to construct
为了赋予 ReaRAG 利用外部知识构建推理链的能力
reasoning chain guided by external knowledge, we propose an automated data construction approach, as detailed in Algorithm 1.
，我们提出了一种自动化数据构建方法，详细描述见算法 1。

Data construction. Given a multi-hop question $x_i$$x_i$x_(i)x_{i} sampled from the seed dataset, we prompt the LRM ${\mathcal{M}}_{LRM}$${\mathcal{M}}_{LRM}$M_(LRM)\mathcal{M}_{L R M} with instruction prompt ${\mathcal{P}}_d$${\mathcal{P}}_d$P_(d)\mathcal{P}_{d} (see Appendix B ) to collect the reasoning thoughts and actions. Next, the search query is extracted and executed against RAG Engine $\mathcal{R}$$\mathcal{R}$R\mathcal{R} to obtain an observation $o_t$$o_t$o_(t)o_{t}. The process iterates until either model decides an finish action or the iteration count exceeds the maximum iterations $T_{\text{max}}$$T_{\text{max }}$T_("max ")T_{\text {max }}.
数据构建。给定从种子数据集中抽样的多跳问题 $x_i$$x_i$x_(i)x_{i} ，我们使用指令提示 ${\mathcal{P}}_d$${\mathcal{P}}_d$P_(d)\mathcal{P}_{d} （参见附录 B）提示 LRM ${\mathcal{M}}_{LRM}$${\mathcal{M}}_{LRM}$M_(LRM)\mathcal{M}_{L R M} ，以收集推理思路和动作。接下来，提取搜索查询并针对 RAG 引擎 $\mathcal{R}$$\mathcal{R}$R\mathcal{R} 执行，以获取观察结果 $o_t$$o_t$o_(t)o_{t} 。该过程会迭代进行，直到模型决定完成动作或迭代次数超过最大迭代次数 $T_{\text{max}}$$T_{\text{max }}$T_("max ")T_{\text {max }} 。

Data filtering. Previous work studies have shown that the performance of the LLMs heavily depends on the quality of fine-tuning data (Gunasekar et al., 2023). To ensure high-quality reasoning chains, we apply data filtering by comparing the final answer ${\hat{y}}_i$${\widehat{y}}_i$hat(y)_(i)\hat{y}_{i} derived from the reasoning chain ${\mathcal{C}}_i$${\mathcal{C}}_i$C_(i)\mathcal{C}_{i} against the ground truth answer $y_i$$y_i$y_(i)y_{i} using the F1 metric. Reasoning chains with an F1 score of 0 are discarded to maintain data integrity.
数据过滤。之前的研究工作表明，LLMs 的性能高度依赖于微调数据的质量（Gunasekar 等，2023）。为确保高质量的推理链，我们通过使用 F1 指标比较从推理链 ${\mathcal{C}}_i$${\mathcal{C}}_i$C_(i)\mathcal{C}_{i} 得出的最终答案 ${\hat{y}}_i$${\widehat{y}}_i$hat(y)_(i)\hat{y}_{i} 和基准真值答案 $y_i$$y_i$y_(i)y_{i} ，来进行数据过滤。F1 分数为 0 的推理链将被丢弃，以保持数据完整性。

Fine-tuning. To incorporate knowledge-guided reasoning ability into the model, we perform supervised fine-tuning (SFT) on the constructed dataset discussed in the previous section. Each sample is a sequence of conversation chain $\mathcal{S}=\{\mathcal{P},x_i,{\left\{({\tau }_t,{\alpha }_t,o_t)\right\}}_{t=1}^N\}$$\mathcal{S}=\left\{\mathcal{P},x_i,{\left\{\left({\tau }_t,{\alpha }_t,o_t\right)\right\}}_{t=1}^N\right\}$S={P,x_(i),{(tau_(t),alpha_(t),o_(t))}_(t=1)^(N)}\mathcal{S}=\left\{\mathcal{P}, x_{i},\left\{\left(\tau_{t}, \alpha_{t}, o_{t}\right)\right\}_{t=1}^{N}\right\}, where $\mathcal{P}$$\mathcal{P}$P\mathcal{P} denotes the instruction prompt (see Appendix B). We fine-tune our factuality-enhanced LRM, ReaRAG ( ${\mathcal{M}}_{\text{ReaRAG}}$${\mathcal{M}}_{\text{ReaRAG }}$M_("ReaRAG ")\mathcal{M}_{\text {ReaRAG }} ) using loss function below:
微调。为将知识引导的推理能力融入模型，我们对上一节中构建的数据集进行监督微调（SFT）。每个样本是一个会话链序列 $\mathcal{S}=\{\mathcal{P},x_i,{\left\{({\tau }_t,{\alpha }_t,o_t)\right\}}_{t=1}^N\}$$\mathcal{S}=\left\{\mathcal{P},x_i,{\left\{\left({\tau }_t,{\alpha }_t,o_t\right)\right\}}_{t=1}^N\right\}$S={P,x_(i),{(tau_(t),alpha_(t),o_(t))}_(t=1)^(N)}\mathcal{S}=\left\{\mathcal{P}, x_{i},\left\{\left(\tau_{t}, \alpha_{t}, o_{t}\right)\right\}_{t=1}^{N}\right\} ，其中 $\mathcal{P}$$\mathcal{P}$P\mathcal{P} 表示指令提示（参见附录 B）。我们使用下面的损失函数对我们的事实性增强 LRM ReaRAG ${\mathcal{M}}_{\text{ReaRAG}}$${\mathcal{M}}_{\text{ReaRAG }}$M_("ReaRAG ")\mathcal{M}_{\text {ReaRAG }} 进行微调：

where $s_j$$s_j$s_(j)s_{j} represents the textual tokens of the input sequence $\mathcal{S},\mathbf{1}(\cdot )$$\mathcal{S},\mathbf{1}(\cdot )$S,1(*)\mathcal{S}, \mathbf{1}(\cdot) is a loss mask indicator, which is set to True on thought ${\tau }_t$${\tau }_t$tau_(t)\tau_{t} and action ${\alpha }_t$${\alpha }_t$alpha_(t)\alpha_{t} tokens, ensuring that the loss is only computed over the tokens contributed to the thought reasoning as well as action, rather than the entire sequence.
其中 $s_j$$s_j$s_(j)s_{j} 表示输入序列的文本标记， $\mathcal{S},\mathbf{1}(\cdot )$$\mathcal{S},\mathbf{1}(\cdot )$S,1(*)\mathcal{S}, \mathbf{1}(\cdot) 是损失掩码指示器，在思考 ${\tau }_t$${\tau }_t$tau_(t)\tau_{t} 和动作 ${\alpha }_t$${\alpha }_t$alpha_(t)\alpha_{t} 标记上设置为 True，确保损失仅在对思考推理和动作有贡献的标记上计算，而不是整个序列。

Inference. After fine-tuning, ReaRAG is equipped with advanced reasoning capabilities to solve multi-hop QA. Given an instruction prompt $\mathcal{P}$$\mathcal{P}$P\mathcal{P} and a question $x$$x$xx, the model first generates a reasoning thought ${\tau }_0$${\tau }_0$tau_(0)\tau_{0} and an initial action ${\alpha }_0$${\alpha }_0$alpha_(0)\alpha_{0}, typically a search action. The search query $q_s$$q_s$q_(s)q_{s} is extracted and executed over the RAG engine $\mathcal{R}$$\mathcal{R}$R\mathcal{R}, which returns an observation $o_0$$o_0$o_(0)o_{0}. The triplet
推理。经过微调，ReaRAG 具备了解决多跳问答的高级推理能力。给定指令提示 $\mathcal{P}$$\mathcal{P}$P\mathcal{P} 和问题 $x$$x$xx ，模型首先生成推理思路 ${\tau }_0$${\tau }_0$tau_(0)\tau_{0} 和初始动作 ${\alpha }_0$${\alpha }_0$alpha_(0)\alpha_{0} ，通常是搜索动作。搜索查询 $q_s$$q_s$q_(s)q_{s} 被提取并在 RAG 引擎 $\mathcal{R}$$\mathcal{R}$R\mathcal{R} 上执行，返回一个观察结果 $o_0$$o_0$o_(0)o_{0} 。这个三元组

Algorithm 2 Inference
Input: Input question \(x\), documents \(d o c\), ReaRAG
\(\mathcal{M}_{\text {Rearag }}\), Answer LLM \(\mathcal{M}_{\text {Ans }}\), Instruction
prompt \(\mathcal{P}\), Answer Prompt \(\mathcal{P}_{\text {ans }}\), Max iterations
\(T_{\text {max }}\), RAG engine \(\mathcal{R}\)
Output: Final answer, \(\hat{y}\)
    \(t \leftarrow 0, \mathcal{C} \leftarrow[]\) Sum \(\leftarrow[] \quad \triangleright\) Initialization
    while \(t<T_{\text {max }}\) do
        \(y_{t}^{\prime} \leftarrow \mathcal{M}_{\text {Rearag }}([\mathcal{P} \oplus x \oplus \mathcal{C}])\)
                \(\triangleright\) Generate response for iteration \(t\)
        \(\left(\tau_{t}, \alpha_{t}\right) \leftarrow \operatorname{parse}\left(y_{t}^{\prime}\right)\)
                \(\triangleright\) Extract thought \(\tau_{t}\) and action \(\alpha_{t}\)
        Get action type \(\alpha_{t_{\text {type }}} \leftarrow \alpha_{t}\) ['type']
        if finish \(\in \alpha_{t_{\text {type }}}\) then
            \(\hat{y} \leftarrow \mathcal{M}_{\text {Ans }}\left(\left[\mathcal{P}_{\text {ans }} \oplus x \oplus S u m\right]\right)\)
            return final answer \(\hat{y}\)
        else if search \(\in \alpha_{t_{\text {type }}}\) then
            \(q_{s} \leftarrow \alpha_{t}\) ['query']
            \(o_{t} \leftarrow \mathcal{R}\left(q_{s}, d o c\right)\)
                    \(\triangleright\) Get \(o_{t}\) from RAG engine
            Append ( \(\tau_{t}, \alpha_{t}, o_{t}\) ) to \(\mathcal{C}\)
            Append ( \(q_{s}, o_{t}\) ) to Sum
        end if
        \(t \leftarrow t+1\)
    end while

( ${\tau }_t,{\alpha }_t,o_t$${\tau }_t,{\alpha }_t,o_t$tau_(t),alpha_(t),o_(t)\tau_{t}, \alpha_{t}, o_{t} ) is appended to the reasoning chain $\mathcal{C}$$\mathcal{C}$C\mathcal{C}, while the pair ( $q_s,o_t$$q_s,o_t$q_(s),o_(t)q_{s}, o_{t} ) is appended to the summary chain Sum. This process iterates until ReaRAG decides on a finish action at step $N$$N$NN. Upon this action, we concatenate all gathered information in Sum and prompt an answer model ${\mathcal{M}}_{\text{Ans}}$${\mathcal{M}}_{\text{Ans }}$M_("Ans ")\mathcal{M}_{\text {Ans }} to generate final answer $\hat{y}$$\widehat{y}$hat(y)\hat{y} using the prompt ${\mathcal{P}}_{\text{ans}}$${\mathcal{P}}_{\text{ans }}$P_("ans ")\mathcal{P}_{\text {ans }} described in Appendix B. The pseudocode for the inference stage is provided in Algorithm 2.
（ ${\tau }_t,{\alpha }_t,o_t$${\tau }_t,{\alpha }_t,o_t$tau_(t),alpha_(t),o_(t)\tau_{t}, \alpha_{t}, o_{t} ）被附加到推理链 $\mathcal{C}$$\mathcal{C}$C\mathcal{C} ，同时对（ $q_s,o_t$$q_s,o_t$q_(s),o_(t)q_{s}, o_{t} ）对被附加到总结链 Sum 中。这个过程会迭代进行，直到 ReaRAG 在步骤 $N$$N$NN 决定结束操作。在此操作后，我们将 Sum 中收集的所有信息进行连接，并提示答案模型 ${\mathcal{M}}_{\text{Ans}}$${\mathcal{M}}_{\text{Ans }}$M_("Ans ")\mathcal{M}_{\text {Ans }} 使用附录 B 中描述的提示语 ${\mathcal{P}}_{\text{ans}}$${\mathcal{P}}_{\text{ans }}$P_("ans ")\mathcal{P}_{\text {ans }} 生成最终答案 $\hat{y}$$\widehat{y}$hat(y)\hat{y} 。推理阶段的伪代码在算法 2 中提供。

Dataset and metrics. We evaluate our approach on multi-hop QA benchmarks requiring reasoning across multiple documents, including MuSiQue (MQ) (Trivedi et al., 2022), HotpotQA (HP) (Yang et al., 2018) and IIRC (Ferguson et al., 2020), as well as recently proposed benchmarks, including FRAMES (FR) (Krishna et al., 2024) and FanOutQA (FQ) (Zhu et al., 2024). To evaluate single-hop reasoning capabilities, we additionally include NQ (Kwiatkowski et al., 2019). Since these datasets require open-ended answers, traditional metric such as exact match (EM) may fail to cap-
数据集和指标。我们在需要跨多个文档进行推理的多跳问答基准上评估了我们的方法，包括 MuSiQue (MQ)（Trivedi 等，2022），HotpotQA (HP)（Yang 等，2018）和 IIRC（Ferguson 等，2020），以及最近提出的基准，包括 FRAMES (FR)（Krishna 等，2024）和 FanOutQA (FQ)（Zhu 等，2024）。为了评估单跳推理能力，我们还额外包括了 NQ（Kwiatkowski 等，2019）。由于这些数据集需要开放式答案，传统指标（如精确匹配（EM））可能无法
ture semantically equivalent responses (Yin et al., 2024). Hence, we include the LLM-as-a-Judge metric ( ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} ) (Zheng et al., 2023) based on GPT4 o for more accurate evaluation using the prompt ${\mathcal{P}}_{judge}$${\mathcal{P}}_{judge}$P_(judge)\mathcal{P}_{j u d g e} described in Appendix B. For FanOutQA, we follow the metric used in the original benchmark and report the Rouge-L (R-L) score instead of EM. We randomly sample 100 validation examples each from MuSiQue, HotpotQA, and NQ, 200 from IIRC, and 396 from FRAMES. For FanOutQA, we evaluate on the full development set, which contains 310 examples.
为了获得语义等效的响应（Yin 等人，2024），我们引入了基于 GPT4 的 LLM-as-a-Judge 指标（ ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} ）（Zheng 等人，2023），以使用附录 B 中描述的提示词 ${\mathcal{P}}_{judge}$${\mathcal{P}}_{judge}$P_(judge)\mathcal{P}_{j u d g e} 进行更准确的评估。对于 FanOutQA，我们遵循原始基准测试中使用的指标，并报告 Rouge-L（R-L）分数，而非精确匹配（EM）分数。我们从 MuSiQue、HotpotQA 和 NQ 中随机抽样 100 个验证示例，从 IIRC 中抽样 200 个，从 FRAMES 中抽样 396 个。对于 FanOutQA，我们在包含 310 个示例的完整开发集上进行评估。

Baselines. We compare our approach against multiple baselines, categorized based on their access to external knowledge. These include incontext retrieval, where the corpus is directly appended to the language model’s context; vanilla $RAG$$RAG$RAGR A G, which performs a single retrieval based on the original input question; and state-of-the-art $ad$$ad$ada d vanced RAG methods proposed recently.
基准测试。我们将我们的方法与多个基准进行比较，这些基准根据其对外部知识的访问方式进行分类。这些基准包括：上下文检索，即直接将语料库附加到语言模型的上下文中；原始 $RAG$$RAG$RAGR A G ，它基于原始输入问题执行单次检索；以及最近提出的最先进的 $ad$$ad$ada d 高级 RAG 方法。

In the in-context settings, we use GLM-4-9B and GLM-4-32B (Zeng et al., 2024), both with a 128 k context window. The vanilla $RAG$$RAG$RAGR A G baselines employ the same models, along with QwQ32B (Team, 2024). For advanced $RAG$$RAG$RAGR A G baselines, we include Self-RAG (Asai et al., 2024), which fine-tunes Llama-2-7B to retrieve documents on demand and filter noisy evidence; SearChain (Xu et al., 2024), which generates a Chain-of-Query structure enabling multi-turn retrieval with verification; and Search-o1 (Li et al., 2025), which proposes a framework for LRMs to perform iterative knowledge retrieval. We further consider R1Searcher (Song et al., 2025) and ReSearch (Chen et al., 2025), which incentivize LLMs to perform reasoning prior to retrieval through RL.
在上下文设置中，我们使用 GLM-4-9B 和 GLM-4-32B（Zeng 等，2024），两者都具有 128k 上下文窗口。原始的 $RAG$$RAG$RAGR A G 基线采用相同的模型，以及 QwQ32B（Team，2024）。对于高级的 $RAG$$RAG$RAGR A G 基线，我们包括 Self-RAG（Asai 等，2024），它微调 Llama-2-7B 以按需检索文档并过滤噪声证据；SearChain（Xu 等，2024），它生成查询链结构，支持多轮检索和验证；以及 Search-o1（Li 等，2025），它为 LRMs 提出了执行迭代知识检索的框架。我们进一步考虑 R1Searcher（Song 等，2025）和 ReSearch（Chen 等，2025），这些方法通过强化学习激励 LLMs 在检索之前进行推理。

Data construction and fine-tuning. The seed dataset described in Algorithm 1 is derived from the training sets of MuSiQue, HotpotQA, and NQ, with QwQ-32B as the LRM. To ensure model’s general capabilities, we fine-tune GLM-4-9B (Zeng et al., 2024) with the constructed dataset (roughly 20k filtered samples), as well as the general SFT dataset from GLM-4 (Zeng et al., 2024).
数据构建和微调。算法 1 中描述的种子数据集源于 MuSiQue、HotpotQA 和 NQ 的训练集，以 QwQ-32B 作为 LRM。为确保模型的通用能力，我们使用构建的数据集（约 2 万个筛选样本）以及 GLM-4 的通用 SFT 数据集对 GLM-4-9B（Zeng 等，2024）进行微调。

Evaluation. For MuSiQue and HotpotQA, we use the original corpora provided by the respective authors. For NQ, we follow the corpus setup from “Lost in the middle” (Liu et al., 2024). To increase the difficulty, particularly for comparisons with
评估。对于 MuSiQue 和 HotpotQA，我们使用各作者提供的原始语料库。对于 NQ，我们遵循"迷失在中间"（Liu 等，2024）的语料库设置。为了增加难度，特别是为了与

Table 1: Main experimental results across six benchmarks. Bold and underline indicate the best and second best results. We report the traditional EM scores as well as ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L}, a metric based on the LLM-as-a-Judge framework using GPT-4o. For FanOutQA, we follow the original benchmark and report the Rouge-L (R-L) score. Our model, ReaRAG-9B, achieves the best overall performance, demonstrating the effectiveness of our strategic distillation approach, even when compared to recent RL-based baselines that integrate reasoning with retrieval.
表 1：六个基准测试上的主要实验结果。粗体和下划线分别表示最佳和次佳结果。我们报告了传统的精确匹配（EM）分数以及基于 GPT-4o 的 LLM 作为评判者框架的 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} 指标。对于 FanOutQA，我们遵循原始基准测试并报告 Rouge-L（R-L）分数。我们的模型 ReaRAG-9B 实现了最佳整体性能，证明了我们的战略蒸馏方法的有效性，即使与最近集成推理和检索的基于强化学习的基准模型相比。
long-context LLMs, we scale up the number of distractor documents, resulting in corpora with 48k to 58 k token lengths. Thus, this design demands high-quality queries to enhance retrieval quality.
对于长上下文 LLM，我们将干扰文档的数量扩大到 48k 到 58k 个标记长度。因此，这种设计需要高质量的查询以提高检索质量。

For IIRC, FRAMES, and FanOutQA, where official corpora are incomplete or unavailable, we retrieve the top 5 most relevant Wikipedia snippets via the Google Serper API. These benchmarks are excluded from the in-context settings evaluations due to the lack of official corpora. All baselines are evaluated using their official implementations with our RAG engine and corpus setup.
对于 IIRC、FRAMES 和 FanOutQA，由于官方语料库不完整或不可用，我们通过 Google Serper API 检索了前 5 个最相关的维基百科片段。由于缺乏官方语料库，这些基准测试被排除在上下文设置评估之外。所有基线都使用其官方实现和我们的 RAG 引擎以及语料库设置进行评估。

RAG engine. Our RAG engine consists of two main components: retrieval and generation. For retrieval, we utilize the embedding model embedding3 from Zhipu’s API ${}^3$${}^3$^(3){ }^{3}, along with a reranker based on the GLM3 architecture to enhance retrieval quality. This setup is applied only to benchmarks with self-constructed corpora, while for IIRC, FRAMES, and FanOutQA, documents are retrieved directly via Google. For generation, we use GLM-4-32B with a 128 k context window to generate responses based on the retrieved documents.
RAG 引擎。我们的 RAG 引擎由两个主要组件组成：检索和生成。在检索方面，我们利用智谱 API ${}^3$${}^3$^(3){ }^{3} 的嵌入模型 embedding3，以及基于 GLM3 架构的重新排序器来提高检索质量。这种设置仅适用于自构建语料库的基准测试，而对于 IIRC、FRAMES 和 FanOutQA，文档直接通过 Google 检索。对于生成，我们使用具有 128k 上下文窗口的 GLM-4-32B 根据检索到的文档生成响应。

Table 1 presents our main results across six benchmarks. Overall, our approach ReaRAG9 B achieves the highest average ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} (48.5)
表 1 展示了我们在六个基准测试中的主要结果。总体而言，我们的方法 ReaRAG9 B 实现了最高的平均 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} （48.5）

across all evaluated methods. Despite the recent momentum around RL in RAG, particularly with models such as R1-Searcher and ReSearch, ReaRAG-9B trained solely with SFT to generate knowledge-guided reasoning chain, still matches or exceeds performance on MuSiQue, HotpotQA, IIRC, and NQ, and outperforms RL-based baselines on FRAMES and FanOutQA (by $5.8\mathrm{\%}-7.3\mathrm{\%}$$5.8\mathrm{\%}-7.3\mathrm{\%}$5.8%-7.3%5.8 \%-7.3 \% ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} ), suggesting SFT can achieve equally powerful performance for open-ended QA tasks. Although designed for multi-hop tasks, ReaRAG also performs strongly on single-hop QA, achieving 55.5 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} on NQ, demonstrating ReaRAG’s capabilities beyond multi-hop reasoning.
在所有评估的方法中。尽管最近围绕 RAG 中的强化学习（RL）有动力，特别是像 R1-Searcher 和 ReSearch 这样的模型，但仅使用监督微调（SFT）训练的 ReaRAG-9B，在生成知识引导的推理链时，仍然在 MuSiQue、HotpotQA、IIRC 和 NQ 上匹配或超越性能，并且在 FRAMES 和 FanOutQA 上优于基于强化学习的基线（通过 $5.8\mathrm{\%}-7.3\mathrm{\%}$$5.8\mathrm{\%}-7.3\mathrm{\%}$5.8%-7.3%5.8 \%-7.3 \% ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} ），这表明 SFT 可以为开放式问答任务实现同样强大的性能。尽管设计用于多跳任务，但 ReaRAG 在单跳问答上也表现出色，在 NQ 上达到 55.5 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} ，展示了 ReaRAG 超越多跳推理的能力。

However, discrepancies between EM and ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L}, e.g., on NQ, where ReaRAG achieves a higher ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} than ReSearch ( 55.5 vs . 53.0) but yields a lower EM score ( 32.0 vs. 37.0), suggest that EM may fail to capture contextually valid answers, a phenomenon consistently observed across all other benchmarks, necessitating the use of ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} metric.
然而，精确匹配（EM）和 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} 之间的差异，例如在 NQ 上，ReaRAG 的 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} 高于 ReSearch（55.5 比 53.0），但 EM 分数较低（32.0 比 37.0），表明 EM 可能无法捕捉上下文有效的答案，这是在所有其他基准测试中一致观察到的现象，因此需要使用 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} 指标。

Comparing vanilla RAG with in-context settings across different GLM-4 backbone scales, we find that vanilla RAG generally performs better, suggesting that long-context models may struggle with distractor-heavy corpora. The only exception is MuSiQue with the GLM-4-32B backbone, where the in-context setting slightly outperforms vanilla $RAG$$RAG$RAGR A G ( 33.50 vs. 29.00 ). Under vanilla $RAG$$RAG$RAGR A G setting, QwQ-32B significantly outperforms GLM-4-
在不同的 GLM-4 骨干规模上比较传统 RAG 和上下文设置，我们发现传统 RAG 通常表现更好，这表明长上下文模型可能难以处理干扰词丰富的语料库。唯一的例外是使用 GLM-4-32B 骨干的 MuSiQue，其中上下文设置略微优于传统 $RAG$$RAG$RAGR A G （33.50 比 29.00）。在传统 $RAG$$RAG$RAGR A G 设置下，QwQ-32B 在 FRAMES 和 FanOutQA 等复杂的多跳基准测试中显著优于 GLM-4-32B，突显了具有强大推理能力的大型语言模型（LRMs）的优势。

32B on complex multi-hop benchmarks such as FRAMES and FanOutQA, highlighting the advantage of LRMs with strong reasoning capabilities.
32B 在复杂的多跳基准测试中表现出色，突显了具有强大推理能力的大型语言模型（LRMs）的优势。

Notably, the original SearChain paper reports results using GPT-3.5, a proprietary large-scale model. For a fair comparison, we replace it with Qwen2.5-7B-Instruct (Yang et al., 2024). Under this setting, SearChain underperforms across all benchmarks. Similarly, Self-RAG, despite aiming to improve retrieval quality, lacks multi-turn retrieval strategies and shows weak performance. Search-o1, although leveraging QwQ-32B, a strong reasoning LRM to perform multi-turn retrieval, only performs on par with vanilla $RAG$$RAG$RAGR A G settings. We further analyze these findings in Section 4.5.
值得注意的是，原始的 SearChain 论文报告了使用 GPT-3.5（一个专有的大规模模型）的结果。为了进行公平的比较，我们将其替换为 Qwen2.5-7B-Instruct（Yang 等，2024）。在这种设置下，SearChain 在所有基准测试中表现不佳。同样，Self-RAG 尽管旨在改进检索质量，但缺乏多轮检索策略，并显示出较弱的性能。Search-o1 虽然利用了 QwQ-32B（一个强大的推理大语言模型）来执行多轮检索，但仅与原始 $RAG$$RAG$RAGR A G 设置相当。我们将在第 4.5 节进一步分析这些发现。

Closed-book performance. We conduct a closed-book experiment to evaluate the parametric knowledge of the language models. The results, presented in Table 3 show that QwQ-32B outperforms GLM-4 on multi-hop benchmarks requiring strong reasoning, except for HotpotQA and FanOutQA. Nevertheless, their parametric knowledge remains insufficient compared to the results in Table 1.
闭卷性能。我们进行了一个闭卷实验，以评估语言模型的参数化知识。表 3 中的结果显示，QwQ-32B 在需要强大推理能力的多跳基准测试中优于 GLM-4，除了 HotpotQA 和 FanOutQA。尽管如此，与表 1 中的结果相比，他们的参数化知识仍然不足。

Advantage of strong reasoning. To evaluate the impact of strong reasoning capabilities, we finetune a model that lacks such abilities under the same Thought-Action-Observation paradigm. This variant, denoted as w/o reasoning in Table 4, shares the same backbone architecture as ReaRAG-9B and follows the data construction process outlined in Algorithm 3. However, instead of leveraging a strong reasoning model like QwQ-32B for data generation, we employ GLM-4-9B, which lacks robust reasoning abilities. Unlike the previous data construction approach in Algorithm 1, which used only multi-hop questions as input, we now provide GLM-4-9B with additional information, including ground-truth decompositions and ground-truth answers. The instruction prompt ${\mathcal{P}}_{\text{ablat}}$${\mathcal{P}}_{\text{ablat }}$P_("ablat ")\mathcal{P}_{\text {ablat }} used to generate its reasoning chain is detailed in Appendix B.
强推理能力的优势。为了评估强推理能力的影响，我们对缺乏这种能力的模型进行微调，同时遵循思考-行动-观察范式。如表 4 中标记为"无推理"的变体，其骨干架构与 ReaRAG-9B 相同，并遵循算法 3 中概述的数据构建过程。然而，与之前使用多跳问题作为输入的算法 1 不同，我们现在为 GLM-4-9B 提供额外信息，包括地面真实的分解和地面真实的答案。用于生成其推理链的指令提示 ${\mathcal{P}}_{\text{ablat}}$${\mathcal{P}}_{\text{ablat }}$P_("ablat ")\mathcal{P}_{\text {ablat }} 详见附录 B。

Table 4 shows that ReaRAG-9B with enhanced reasoning capabilities (w/ reasoning) consistently outperforms its counterpart without reasoning, achieving a notable gain of 6-12% ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} gain on the multi-hop benchmarks and $3.5\mathrm{\%}$$3.5\mathrm{\%}$3.5%3.5 \% gain on singlehop NQ. The smaller gain on NQ suggests that strong reasoning offers limited benefit for singlehop tasks, consistent with the assumption that such
表 4 显示，具有增强推理能力的 ReaRAG-9B（带推理）始终优于没有推理的对应模型，在多跳基准测试中实现了 6-12%的显著提升 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} ，并在单跳 NQ 任务中获得了 $3.5\mathrm{\%}$$3.5\mathrm{\%}$3.5%3.5 \% 的提升。在 NQ 上的较小提升表明强大的推理能力对单跳任务的帮助有限，这与这种假设是一致的
questions require less compositional reasoning.
问题需要较少的组合推理能力。

Table 2: Invalid generation rates of special tokens in QwQ-32B, leading to retrieval failures in Search-o1.
表 2：QwQ-32B 中特殊标记的无效生成率，导致搜索-o1 中的检索失败

We conduct an in-depth analysis and compare our approach against the strong baselines, including Search-o1, R1-Searcher and ReSearch. Below, we highlight key factors affecting their performance.
我们对我们的方法进行了深入分析，并将其与强基线方法进行比较，包括 Search-o1、R1-Searcher 和 ReSearch。下面，我们重点阐述影响其性能的关键因素。

Token generation failure. $\mathrm{QwQ}-32\mathrm{B}$$\mathrm{QwQ}-32\mathrm{B}$QwQ-32B\mathrm{QwQ}-32 \mathrm{~B} struggles to follow the instruction prompt, failing to generate special tokens (e.g., |begin_search_query|) essential for Search-o1 to retrieve external knowledge. This limitation forces Search-o1 into a closed-book setting, significantly impairing its performance. Table 2 quantifies this issue, revealing invalid generation rates of 18-30%.
令牌生成失败。 $\mathrm{QwQ}-32\mathrm{B}$$\mathrm{QwQ}-32\mathrm{B}$QwQ-32B\mathrm{QwQ}-32 \mathrm{~B} 无法遵循指令提示，未能生成对 Search-o1 检索外部知识至关重要的特殊令牌（例如 |begin_search_query|）。这一限制迫使 Search-o1 陷入封闭书本设置，显著损害其性能。表 2 量化了这一问题，揭示了 18-30% 的无效生成率。

Information extraction failure. Search-o1 introduces the Reason-in-Documents module, leveraging QwQ-32B for in-depth reasoning over retrieved documents to generate responses as search results. However, this module has a key limitation: it may incorrectly conclude that no useful information is available (Table 5). Our analysis identifies the primary cause: the module attempts to answer the original multi-hop question based on the search query, but the retrieved information is insufficient. For example, as shown in Table 5, the module searches for information related to “Hannibal and Scipio book” at first search, but the retrieved content only includes the book’s author, lacking information about the place of education. This flaw weakens Search-o1, as it continuously searches for non-existent information, causing reasoning paths to diverge and ultimately hitting iteration limits.
信息提取失败。Search-o1 引入了"文档推理"模块，利用 QwQ-32B 对检索到的文档进行深入推理以生成搜索结果。然而，该模块存在一个关键限制：可能错误地得出没有可用信息的结论（表 5）。我们的分析确定了主要原因：该模块试图基于搜索查询回答原始多跳问题，但检索到的信息不充分。例如，如表 5 所示，模块首次搜索与"汉尼拔和斯基皮奥书籍"相关的信息，但检索到的内容仅包含书籍作者，缺乏关于教育地点的信息。这一缺陷削弱了 Search-o1，因为它持续搜索不存在的信息，导致推理路径发散并最终达到迭代限制。

Hallucination in Reason-in-Documents module. The Reason-in-Documents module is prone to hallucination Table 6. For instance, when searching for the members of Bruce Lee Band, the module fails to find relevant information and fabricates “Less Than Records” based on parametric knowledge rather than the provided corpus. This hallu-
"文档推理"模块中的幻觉。"文档推理"模块容易产生幻觉（表 6）。例如，在搜索布鲁斯·李乐队成员时，模块未能找到相关信息，并基于参数知识而非提供的语料库虚构了"Less Than Records"。这种幻觉-

Figure 3: Comparison of chain length across all benchmarks. We measure the number of retrievals needed for the baselines to achieve a full ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} score. Search-o1 consistently requires more steps than ReaRAG, highlighting the tendency of the underlying LRM to overthink in multi-hop QA tasks.
图 3：所有基准测试中链长度的比较。我们测量基线模型为达到完整 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} 分数所需的检索次数。Search-o1 始终需要比 ReaRAG 更多的步骤，突显了底层 LRM 在多跳问答任务中过度思考的倾向。
cination propagates through subsequent reasoning steps, degrading the final answer quality.
幻觉会通过后续的推理步骤传播，降低最终答案的质量。

Poor generalization on harder benchmarks. R1-Searcher and ReSearch show weaker performance on FRAMES and FanOutQA, despite matching ReaRAG on other benchmarks. Case studies in Tables 7 and 8 reveal that both models struggle to reason effectively in the presence of noisy information. R1-Searcher, in particular, proceeds to generate final answers despite lacking key information. These findings suggest that RL-based models lack the ability to reflect and revise earlier reasoning, an ability essential for robust multi-hop reasoning. In contrast, ReaRAG successfully realigns its reasoning path, even after early-stage errors, demonstrating greater robustness on complex multi-hop tasks.
在更难的基准测试中泛化能力较差。R1-Searcher 和 ReSearch 在 FRAMES 和 FanOutQA 上表现较弱，尽管在其他基准测试中能与 ReaRAG 相匹配。表 7 和表 8 中的案例研究揭示，这两个模型在存在噪音信息时难以有效推理。特别是 R1-Searcher，即便缺乏关键信息也会继续生成最终答案。这些发现表明基于强化学习的模型缺乏反思和修正早期推理的能力，而这对于鲁棒的多跳推理至关重要。相比之下，ReaRAG 能够成功重新调整其推理路径，即使在早期出现错误，在复杂的多跳任务中也表现出更强的鲁棒性。

Overthinking in multi-hop QA. Recent studies have shown that LRMs tend to overthink by generating overly long reasoning chains, causing redundancy in multi-hop QA (Chen et al., 2024; Team et al., 2025). We analyze this by comparing the number of retrievals needed to achieve full ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L}. Compared to Search-o1, ReaRAG requires fewer steps across all benchmarks except for FanOutQA, demonstrating efficiency in multi-hop reasoning while achieving better performance. Case studies in Tables 9 and 10 further highlight this difference. However, for baselines such as ReSearch and R1-Searcher which are optimized for retrieval, ReaRAG consistently takes more steps.
多跳问答中的过度思考。最近的研究表明，大型检索模型（LRM）倾向于通过生成过长的推理链来过度思考，在多跳问答中造成冗余（Chen 等，2024；Team 等，2025）。我们通过比较达到完整 ${\mathrm{ACC}}_L$${\mathrm{ACC}}_L$ACC_(L)\mathrm{ACC}_{L} 所需的检索次数来分析这一点。与 Search-o1 相比，ReaRAG 在除 FanOutQA 之外的所有基准测试中所需步骤更少，展示了在多跳推理中的效率并取得了更好的性能。表 9 和表 10 中的案例研究进一步突显了这一差异。然而，对于像 ReSearch 和 R1-Searcher 这样针对检索进行优化的基线模型，ReaRAG 始终需要更多步骤。

This section showcases ReaRAG’s advanced reasoning capabilities. Table 11 shows that ReaRAG initially mistakenly identified “Anne of Austria” as the grandmother of “Philippe” rather than his mother. However, ReaRAG later detected this mistake, verified the information, and corrected it. This self-correction mechanism helps prevent errors from propagating to later reasoning steps.
本节展示了 ReaRAG 的高级推理能力。表 11 显示，ReaRAG 最初错误地将"奥地利的安妮"误认为是"菲利普"的祖母，而不是他的母亲。然而，ReaRAG 随后检测到了这一错误，验证了信息，并进行了更正。这种自我纠正机制有助于防止错误在后续推理步骤中传播。

Table 12 shows how ReaRAG resolves ambiguity in a multi-hop question via multiple retrieval. The query involves “The Hard Easy”, which refers to both a film and a TV series. At the sixth reasoning step, ReaRAG also faces conflicting information but successfully disambiguates and provides the correct answer.
表 12 展示了 ReaRAG 如何通过多次检索来解决多跳问题中的歧义性。该查询涉及"The Hard Easy"，它指的是一部电影和一部电视剧。在第六个推理步骤中，ReaRAG 还面临着冲突的信息，但成功地消除了歧义并提供了正确的答案。

Table 13 provides another example of ReaRAG handling ambiguity in a multi-hop question while resolving a knowledge conflict. Its parametric knowledge incorrectly states that Sonic is voiced by “Roger Craig Smith” instead of “Jim Cummings”. ReaRAG detects and corrects this inconsistency, ultimately reaching the correct answer. This case further highlights its robust reasoning abilities.
表 13 提供了另一个例子，展示了 ReaRAG 如何处理多跳问题中的歧义性并解决知识冲突。其参数化知识错误地声称声音角色索尼克是由"罗杰·克雷格·史密斯"配音，而不是"吉姆·卡明斯"。ReaRAG 检测并纠正了这一不一致性，最终得出了正确的答案。这一案例进一步突显了其强大的推理能力。

These examples highlight ReaRAG’s ability to iteratively perform knowledge-guided reasoning. Compared to existing baselines, our approach better integrates reasoning model with external knowledge, enhancing factual accuracy.
这些示例突出了 ReaRAG 通过迭代方式执行知识引导推理的能力。与现有基准相比，我们的方法能更好地整合推理模型和外部知识，提高事实准确性。

In this study, we introduce ReaRAG, a factualityenhanced reasoning model capable of performing knowledge-guided reasoning. ReaRAG iteratively plans and reflects on reasoning steps, leveraging external knowledge to validate each step in the reasoning chain. Through comprehensive evaluation across six benchmarks, ReaRAG achieves the best overall performance, outperforming RLbased methods on challenging benchmarks such as FRAMES and FanOutQA, while matching them on easier tasks. Unlike RL-based approaches requiring complex policy optimization and high computational cost, our method strategically distills reasoning capabilities from LRMs, offering a more efficient alternative. Further analysis shows its robustness in handling complex multi-hop questions while mitigating the overthinking behavior seen in Search-o1. These results underscore the effectiveness of strategic reasoning distillation as a practical alternative to RL-based RAG methods.
在本研究中，我们引入了 ReaRAG，这是一个能够执行知识引导推理的事实增强推理模型。ReaRAG 通过迭代方式规划和反思推理步骤，利用外部知识验证推理链中的每一个步骤。通过对六个基准的全面评估，ReaRAG 取得了最佳整体性能，在 FRAMES 和 FanOutQA 等具有挑战性的基准上超越基于强化学习的方法，同时在较简单的任务上与之匹配。与需要复杂策略优化和高计算成本的基于强化学习的方法不同，我们的方法从大型语言模型中战略性地提取推理能力，提供了一种更高效的替代方案。进一步分析表明，该方法在处理复杂的多跳问题时具有鲁棒性，同时减轻了 Search-o1 中观察到的过度思考行为。这些结果凸显了战略性推理蒸馏作为基于强化学习的检索增强生成（RAG）方法的实用替代方案的有效性。

Limited action space While ReaRAG demonstrates strong performance in the QA task, its action space is currently limited to only search and finish in this study. Consequently, it is restricted to processing local knowledge sources and cannot perform actions such as leveraging a code compiler for coding tasks, executing mathematical calculations, or conducting real-time web searches. Expanding its action space could enhance its adaptability across diverse problem domains.
受限的动作空间 虽然 ReaRAG 在问答任务中表现出强大的性能，但在本研究中，其动作空间目前仅限于搜索和完成。因此，它仅限于处理本地知识源，无法执行诸如利用代码编译器进行编码任务、执行数学计算或进行实时网页搜索等操作。扩展其动作空间可以增强其在不同问题领域中的适应性。

Data construction efficiency To equip ReaRAG with a structured reasoning process, we fine-tune ReaRAG using structured responses generated by the LRM. However, this approach relies on the LRM’s strong instruction-following ability, and a substantial portion of the data is discarded due to validity issues, leading to computational inefficiency and resource waste. Improving data augmentation techniques could mitigate this limitation.
数据构建效率 为了使 ReaRAG 具备结构化推理过程，我们使用 LRM 生成的结构化响应对 ReaRAG 进行微调。然而，这种方法依赖于 LRM 的强大指令跟随能力，并且由于有效性问题，大量数据被丢弃，导致计算效率低下和资源浪费。改进数据增强技术可以缓解这一限制。

Inference latency ReaRAG solves questions iteratively, requiring multiple reasoning steps to reach the final answer. While this enhances accuracy, it also increases inference time compared to models that generate answers in a single pass. This trade-off between reasoning depth and efficiency may limit its practicality in real-time applications or scenarios with strict latency constraints.
推理延迟 ReaRAG 通过迭代方式解决问题，需要多个推理步骤来得出最终答案。虽然这提高了准确性，但与单次生成答案的模型相比，它也增加了推理时间。这种推理深度和效率之间的权衡可能会限制其在实时应用或对延迟有严格要求的场景中的实用性。

This work is supported by National Natural Science Foundation of China (62476150), Beijing Natural Science Foundation (L243006), Tsinghua University Initiative Scientific Research Program and Tsinghua University (Department of Computer Science and Technology)-Siemens Ltd., China Joint Research Center for Industrial Intelligence and Internet of Things (JCIIOT).
本项目得到国家自然科学基金（62476150）、北京市自然科学基金（L243006）、清华大学科研创新计划以及清华大学（计算机科学与技术系）-西门子（中国）有限公司工业智能和物联网联合研究中心（JCIIOT）的支持。

This paper involved the use of AI-assisted tools (e.g., ChatGPT) for language refinement and editing. All content was reviewed and verified by the authors.
本论文使用了人工智能辅助工具（如 ChatGPT）进行语言润色和编辑。所有内容均已由作者审查和验证。

Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, and Laurent Sifre. 2022. Improving language models by retrieving from trillions of tokens. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 2206-2240. PMLR.
Sebastian Borgeaud、Arthur Mensch、Jordan Hoffmann、Trevor Cai、Eliza Rutherford、Katie Millican、George van den Driessche、Jean-Baptiste Lespiau、Bogdan Damoc、Aidan Clark、Diego de Las Casas、Aurelia Guy、Jacob Menick、Roman Ring、Tom Hennigan、Saffron Huang、Loren Maggiore、Chris Jones、Albin Cassirer、Andy Brock、Michela Paganini、Geoffrey Irving、Oriol Vinyals、Simon Osindero、Karen Simonyan、Jack W. Rae、Erich Elsen 和 Laurent Sifre. 2022. 通过从数万亿个标记中检索来改进语言模型。载于《机器学习国际会议论文集》（ICML 2022），2022 年 7 月 17-23 日，美国马里兰州巴尔的摩，第 162 卷，第 2206-2240 页。PMLR。

Shulin Cao, Jiajie Zhang, Jiaxin Shi, Xin Lv, Zijun Yao, Qi Tian, Lei Hou, and Juanzi Li. 2023. Probabilistic tree-of-thought reasoning for answering knowledgeintensive complex questions. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 1254112560. Association for Computational Linguistics.
Shulin Cao、Jiajie Zhang、Jiaxin Shi、Xin Lv、Zijun Yao、Qi Tian、Lei Hou 和 Juanzi Li. 2023. 用于回答知识密集型复杂问题的概率思考树推理。载于《计算语言学协会会议成果》（EMNLP 2023），新加坡，2023 年 12 月 6-10 日，第 12541-12560 页。计算语言学协会。

Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, and Jie Fu. 2024. RQ-RAG: learning to refine queries for retrieval augmented generation. CoRR, abs/2404.00610.
Chi-Min Chan、Chunpu Xu、Ruibin Yuan、Hongyin Luo、Wei Xue、Yike Guo 和 Jie Fu. 2024. RQ-RAG：学习优化检索增强生成的查询。CoRR，abs/2404.00610。

Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. 2025. Research: Learning to reason with search for llms via reinforcement learning. CoRR, abs/2503.19470.
Mingyang Chen、Tianpeng Li、Haoze Sun、Yijie Zhou、Chenzheng Zhu、Haofen Wang、Jeff Z. Pan、Wen Zhang、Huajun Chen、Fan Yang、Zenan Zhou 和 Weipeng Chen. 2025. 研究：通过强化学习让 LLMs 学会使用搜索进行推理。CoRR，abs/2503.19470。

Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2024. Do NOT think that much for $2+3=$$2+3=$2+3=2+3= ? on the overthinking of o1-like llms. CoRR, abs/2412.21187.
陈星宇、徐佳豪、梁天、何志伟、庞建辉、余典、宋林峰、刘秋芝、周梦菲、张卓生、王瑞、涂兆鹏、米海涛和余东。2024 年。不要为 $2+3=$$2+3=$2+3=2+3= 想太多？关于 O1 类 LLMs 的过度思考。CoRR，abs/2412.21187。

DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang,
DeepSeek-AI、郭大亚、杨德健、张浩伟、宋俊骁、张若瑜、许润心、朱启豪、马士荣、王培义、毕小、张晓康、余星凯、吴宇、吴泽锋、苟志斌、邵志红、李卓舒、高子怡、刘爱新、薛冰、王斌宣、吴柏超、冯蓓、陆成达、赵成刚、邓成琦、张晨宇、阮崇、戴大美、陈德立、纪东杰、李尔航、林方云、戴福聪、罗福利、郝广博、陈冠廷、李国伟、张海、鲍汉、徐涵未、王浩成、丁红辉、辛华建、高华佐、曲慧、李辉、郭建中、李家世、王佳伟、陈静昌、袁景阳、邱俊杰、李俊龙、蔡建龙、倪佳琪、梁健、陈进、董凯、胡凯、高凯歌、关康、黄科欣、于奎、王乐、张乐聪、赵亮、王立同、张丽悦、徐雷、夏乐怡、张明川、张明华、唐明辉、李萌、王妙君、李明明、田宁、黄盼盼、张鹏、王前程、陈琴宇、杜秋实、葛瑞琦、张瑞松、潘瑞哲、王润基
R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, and S. S. Li. 2025. Deepseek-r1: Incentivizing reasoning capability in 11 ms via reinforcement learning. CoRR, abs/2501.12948.
陈瑞杰、金瑞龙、陈儒易、陆尚豪、周上研、陈善煌、叶胜峰、王世宇、余水平、周顺峰、潘书婷和李森森。2025 年。DeepSeek-R1：通过强化学习在 11 毫秒内激发推理能力。CoRR，abs/2501.12948。

James Ferguson, Matt Gardner, Hannaneh Hajishirzi, Tushar Khot, and Pradeep Dasigi. 2020. IIRC: A dataset of incomplete information reading comprehension questions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 1137-1147. Association for Computational Linguistics.
詹姆斯·弗格森、马特·加德纳、汉娜·哈吉希尔兹、图沙尔·科特和普拉迪普·达西吉。2020 年。IIRC：不完整信息阅读理解问题数据集。载《2020 年自然语言处理实证方法会议论文集》，EMNLP 2020，线上，2020 年 11 月 16-20 日，第 1137-1147 页。计算语言学协会。

Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi Li. 2023. Textbooks are all you need. CoRR, abs/2306.11644.
《沉浸式翻译 - AI 驱动的 PDF 翻译：公式识别，表格识别，多栏支持，一键导出》 %% Suriya Gunasekar、Yi Zhang、Jyoti Aneja、Caio César Teodoro Mendes、Allie Del Giorno、Sivakanth Gopi、Mojan Javaheripi、Piero Kauffmann、Gustavo de Rosa、Olli Saarikivi、Adil Salim、Shital Shah、Harkirat Singh Behl、Xin Wang、Sébastien Bubeck、Ronen Eldan、Adam Tauman Kalai、Yin Tat Lee 和 Yuanzhi Li. 2023. 教科书是你所需要的。CoRR，abs/2306.11644。

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. Retrieval augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 3929-3938. PMLR.
Kelvin Guu、Kenton Lee、Zora Tung、Panupong Pasupat 和 Ming-Wei Chang. 2020. 检索增强语言模型预训练。在第 37 届机器学习国际会议论文集，ICML 2020，2020 年 7 月 13-18 日，虚拟活动，机器学习研究论文集第 119 卷，第 3929-3938 页。PMLR。

Gautier Izacard, Patrick S. H. Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. Atlas: Few-shot learning with retrieval augmented language models. J. Mach. Learn. Res., 24:251:1-251:43.
Gautier Izacard、Patrick S. H. Lewis、Maria Lomeli、Lucas Hosseini、Fabio Petroni、Timo Schick、Jane Dwivedi-Yu、Armand Joulin、Sebastian Riedel 和 Edouard Grave. 2023. Atlas：具有检索增强的语言模型的小样本学习。机器学习研究期刊，24:251:1-251:43。

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applebaum, Angela Jiang, Ashvin Nair, Barret Zoph, Behrooz Ghorbani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman, Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea Voss, Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts, Daniel Kappler, Daniel Levy, Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang, Edmund Wong, Elizabeth Proehl, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang, Felipe Petroski Such, Filippo Raso, Florencia Leoni, Foivos Tsimpourlas, Francis Song, Fred von Lohmann, Freddie Sulit,
Aaron Jaech、Adam Kalai、Adam Lerer、Adam Richardson、Ahmed El-Kishky、Aiden Low、Alec Helyar、Aleksander Madry、Alex Beutel、Alex Carney、Alex Iftimie、Alex Karpenko、Alex Tachard Passos、Alexander Neitz、Alexander Prokofiev、Alexander Wei、Allison Tam、Ally Bennett、Ananya Kumar、Andre Saraiva、Andrea Vallone、Andrew Duberstein、Andrew Kondrich、Andrey Mishchenko、Andy Applebaum、Angela Jiang、Ashvin Nair、Barret Zoph、Behrooz Ghorbani、Ben Rossen、Benjamin Sokolowsky、Boaz Barak、Bob McGrew、Borys Minaiev、Botao Hao、Bowen Baker、Brandon Houghton、Brandon McKinzie、Brydon Eastman、Camillo Lugaresi、Cary Bassin、Cary Hudson、Chak Ming Li、Charles de Bourcy、Chelsea Voss、Chen Shen、Chong Zhang、Chris Koch、Chris Orsinger、Christopher Hesse、Claudia Fischer、Clive Chan、Dan Roberts、Daniel Kappler、Daniel Levy、Daniel Selsam、David Dohan、David Farhi、David Mely、David Robinson、Dimitris Tsipras、Doug Li、Dragos Oprica、Eben Freeman、Eddie Zhang、Edmund Wong、Elizabeth Proehl、Enoch Cheung、Eric Mitchell、Eric Wallace、Erik Ritter、Evan Mays、Fan Wang、Felipe Petroski Such、Filippo Raso、Florencia Leoni、Foivos Tsimpourlas、Francis Song、Fred von Lohmann、Freddie Sulit

Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guillaume Leclerc, Hadi Salman, Haiming Bao, Hao Sheng, Hart Andrin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman, Hyung Won Chung, Ian Kivlichan, Ian O’Connell, Ian Osband, Ignasi Clavera Gilaberte, and Ilge Akkaya. 2024. Openai o1 system card. CoRR, abs/2412.16720.
Geoff Salmon、Giambattista Parascandolo、Gildas Chabot、Grace Zhao、Greg Brockman、Guillaume Leclerc、Hadi Salman、Haiming Bao、Hao Sheng、Hart Andrin、Hessam Bagherinezhad、Hongyu Ren、Hunter Lightman、Hyung Won Chung、Ian Kivlichan、Ian O'Connell、Ian Osband、Ignasi Clavera Gilaberte，以及 Ilge Akkaya。2024 年。OpenAI O1 系统卡。《CoRR》，abs/2412.16720。

Jinhao Jiang, Jiayi Chen, Junyi Li, Ruiyang Ren, Shijie Wang, Wayne Xin Zhao, Yang Song, and Tao Zhang. 2024. Rag-star: Enhancing deliberative reasoning with retrieval augmented verification and refinement. CoRR, abs/2412.12881.
Jinhao Jiang、Jiayi Chen、Junyi Li、Ruiyang Ren、Shijie Wang、Wayne Xin Zhao、Yang Song，以及 Tao Zhang。2024 年。RAG-Star：通过检索增强的验证和细化来增强深思熟虑的推理。《CoRR》，abs/2412.12881。

Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. 2024. Fact, fetch, and reason: A unified evaluation of retrievalaugmented generation. CoRR, abs/2409.12941.
Satyapriya Krishna、Kalpesh Krishna、Anhad Mohananey、Steven Schwarcz、Adam Stambler、Shyam Upadhyay 和 Manaal Faruqui. 2024. 事实、获取和推理：检索增强生成的统一评估。CoRR, abs/2409.12941.

Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. 2024. Training language models to selfcorrect via reinforcement learning. arXiv preprint arXiv:2409.12917.
Aviral Kumar、Vincent Zhuang、Rishabh Agarwal、Yi Su、John D Co-Reyes、Avi Singh、Kate Baumli、Shariq Iqbal、Colton Bishop、Rebecca Roelofs 等. 2024. 通过强化学习训练语言模型自我纠正。arXiv 预印本 arXiv:2409.12917.

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: a benchmark for question answering research. Trans. Assoc. Comput. Linguistics, 7:452466.
Tom Kwiatkowski、Jennimaria Palomaki、Olivia Redfield、Michael Collins、Ankur P. Parikh、Chris Alberti、Danielle Epstein、Illia Polosukhin、Jacob Devlin、Kenton Lee、Kristina Toutanova、Llion Jones、Matthew Kelcey、Ming-Wei Chang、Andrew M. Dai、Jakob Uszkoreit、Quoc Le 和 Slav Petrov. 2019. 自然问题：问答研究的基准。计算语言学协会汇刊, 7:452-466.

Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Patrick S. H. Lewis、Ethan Perez、Aleksandra Piktus、Fabio Petroni、Vladimir Karpukhin、Naman Goyal、Heinrich Küttler、Mike Lewis、Wen-tau Yih、Tim Rocktäschel、Sebastian Riedel 和 Douwe Kiela. 2020. 知识密集型自然语言处理任务的检索增强生成。神经信息处理系统进展 33：2020 年神经信息处理系统年会，NeurIPS 2020，2020 年 12 月 6-12 日，线上。

Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. Search-o1: Agentic searchenhanced large reasoning models. arXiv preprint arXiv:2501.05366.
李小熙、董冠廷、金佳杰、张雨尧、周裕佳、朱宇涛、张沛田和窦志成. 2025. Search-o1: 代理搜索增强的大规模推理模型. arXiv 预印本 arXiv:2501.05366.

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Trans. Assoc. Comput. Linguistics, 12:157-173.
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni 和 Percy Liang. 2024. 迷失在中间：语言模型如何使用长上下文. 计算语言学学报, 12:157-173.

Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès 和 Tatsunori Hashimoto. 2025. s1: 简单的测试时间缩放. arXiv 预印本 arXiv:2501.19393.

OpenAI. 2024. Learning to reason with 11 ms .
OpenAI. 2024. 学习以 11 毫秒推理.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 5687-5711. Association for Computational Linguistics.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith 和 Mike Lewis. 2023. 测量和缩小语言模型中的组合性差距. 在计算语言学协会会议论文集：EMNLP 2023，新加坡，2023 年 12 月 6-10 日，第 5687-5711 页. 计算语言学协会.

Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 9248-9274. Association for Computational Linguistics.
Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan 和 Weizhu Chen. 2023. 通过迭代检索-生成协同增强检索增强型大型语言模型. 在计算语言学协会会议论文集：EMNLP 2023，新加坡，2023 年 12 月 6-10 日，第 9248-9274 页. 计算语言学协会.

Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 31210-31227. PMLR.
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli 和 Denny Zhou. 2023. 大型语言模型很容易被无关上下文分散注意力. 在机器学习国际会议，ICML 2023，2023 年 7 月 23-29 日，夏威夷火奴鲁鲁，机器学习研究论文集，第 202 卷，第 31210-31227 页. PMLR.

Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2024. REPLUG: retrievalaugmented black-box language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pages 8371-8384. Association for Computational Linguistics.
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer 和 Wen-tau Yih. 2024. REPLUG：检索增强型黑盒语言模型. 在 2024 年北美计算语言学协会人类语言技术大会论文集（第 1 卷：长论文），NAACL 2024，墨西哥城，墨西哥，2024 年 6 月 16-21 日，第 8371-8384 页. 计算语言学协会.

Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. CoRR, abs/2408.03314.
Charlie Snell、Jaehoon Lee、Kelvin Xu 和 Aviral Kumar. 2024. 扩展 LLM 测试时计算可能比扩展模型参数更有效. CoRR, abs/2408.03314.

Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and JiRong Wen. 2025. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. CoRR, abs/2503.05592.
Huatong Song、Jinhao Jiang、Yingqian Min、Jie Chen、Zhipeng Chen、Wayne Xin Zhao、Lei Fang 和 JiRong Wen. 2025. R1-searcher：通过强化学习激励 LLM 的搜索能力. CoRR, abs/2503.05592.

Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang, Haotian Yao, Haotian Zhao, Haoyu Lu, Haoze Li, Haozhen Yu, Hongcheng Gao, Huabin Zheng, Huan Yuan, Jia Chen, Jianhang Guo, Jianlin Su, Jianzhou Wang, Jie Zhao, Jin Zhang, Jingyuan Liu, Junjie Yan, Junyan Wu, Lidong Shi, Ling Ye, Longhui Yu, Mengnan Dong, Neo Zhang, Ningchen Ma, Qiwei Pan, Qucheng Gong, Shaowei Liu, Shengling Ma, Shupeng Wei, Sihan Cao, Siying Huang, Tao Jiang,
Kimi 团队，Angang Du、Bofei Gao、Bowei Xing、Changjiu Jiang、Cheng Chen、Cheng Li、Chenjun Xiao、Chenzhuang Du、Chonghua Liao、Chuning Tang、Congcong Wang、Dehao Zhang、Enming Yuan、Enzhe Lu、Fengxiang Tang、Flood Sung、Guangda Wei、Guokun Lai、Haiqing Guo、Han Zhu、Hao Ding、Hao Hu、Hao Yang、Hao Zhang、Haotian Yao、Haotian Zhao、Haoyu Lu、Haoze Li、Haozhen Yu、Hongcheng Gao、Huabin Zheng、Huan Yuan、Jia Chen、Jianhang Guo、Jianlin Su、Jianzhou Wang、Jie Zhao、Jin Zhang、Jingyuan Liu、Junjie Yan、Junyan Wu、Lidong Shi、Ling Ye、Longhui Yu、Mengnan Dong、Neo Zhang、Ningchen Ma、Qiwei Pan、Qucheng Gong、Shaowei Liu、Shengling Ma、Shupeng Wei、Sihan Cao、Siying Huang、Tao Jiang、

Weihao Gao, Weimin Xiong, Weiran He, Weixiao Huang, Wenhao Wu, Wenyang He, Xianghui Wei, Xianqing Jia, Xingzhe Wu, Xinran Xu, Xinxing Zu, Xinyu Zhou, Xuehai Pan, Y. Charles, Yang Li, Yangyang Hu, Yangyang Liu, Yanru Chen, Yejie Wang, Yibo Liu, Yidao Qin, Yifeng Liu, Ying Yang, Yiping Bao, Yulun Du, Yuxin Wu, Yuzhi Wang, Zaida Zhou, Zhaoji Wang, Zhaowei Li, Zhen Zhu, Zheng Zhang, Zhexu Wang, Zhilin Yang, Zhiqi Huang, Zihao Huang, Ziyao Xu, and Zonghan Yang. 2025. Kimi k1.5: Scaling reinforcement learning with 11 ms . CoRR, abs/2501.12599.
Weihao Gao、Weimin Xiong、Weiran He、Weixiao Huang、Wenhao Wu、Wenyang He、Xianghui Wei、Xianqing Jia、Xingzhe Wu、Xinran Xu、Xinxing Zu、Xinyu Zhou、Xuehai Pan、Y. Charles、Yang Li、Yangyang Hu、Yangyang Liu、Yanru Chen、Yejie Wang、Yibo Liu、Yidao Qin、Yifeng Liu、Ying Yang、Yiping Bao、Yulun Du、Yuxin Wu、Yuzhi Wang、Zaida Zhou、Zhaoji Wang、Zhaowei Li、Zhen Zhu、Zheng Zhang、Zhexu Wang、Zhilin Yang、Zhiqi Huang、Zihao Huang、Ziyao Xu 和 Zonghan Yang. 2025. Kimi K1.5：利用 11 毫秒扩展强化学习. CoRR, abs/2501.12599.

Qwen Team. 2024. Qwq: Reflect deeply on the boundaries of the unknown.
Qwen 团队. 2024. Qwq：深入反思未知的边界。

Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multihop questions via single-hop question composition. Trans. Assoc. Comput. Linguistics, 10:539-554.
Harsh Trivedi、Niranjan Balasubramanian、Tushar Khot 和 Ashish Sabharwal. 2022. Musique：通过单跳问题组合的多跳问题. 计算语言学协会汇刊, 10:539-554.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In $Ad$$Ad$AdA d vances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
Jason Wei、Xuezhi Wang、Dale Schuurmans、Maarten Bosma、Brian Ichter、Fei Xia、Ed H. Chi、Quoc V. Le 和 Denny Zhou. 2022. 思维链提示引发大型语言模型的推理. 在 $Ad$$Ad$AdA d 第 35 届神经信息处理系统年会：神经信息处理系统 2022 大会，美国新奥尔良，2022 年 11 月 28 日至 12 月 9 日。

Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, Chenyang Shao, Yuwei Yan, Qinglong Yang, Yiwen Song, Sijian Ren, Xinyuan Hu, Yu Li, Jie Feng, Chen Gao, and Yong Li. 2025. Towards large reasoning models: A survey of reinforced reasoning with large language models. CoRR, abs/2501.09686.
Fengli Xu、Qianyue Hao、Zefang Zong、Jingwei Wang、Yunke Zhang、Jingyi Wang、Xiaochong Lan、Jiahui Gong、Tianjian Ouyang、Fanjin Meng、Chenyang Shao、Yuwei Yan、Qinglong Yang、Yiwen Song、Sijian Ren、Xinyuan Hu、Yu Li、Jie Feng、Chen Gao 和 Yong Li. 2025. 走向大规模推理模型：大型语言模型增强推理的调查. 计算机科学档案, abs/2501.09686.

Shicheng Xu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. 2024. Search-in-the-chain: Interactively enhancing large language models with search for knowledge-intensive tasks. In Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024, pages 1362-1373. ACM.
徐世诚, 庞亮, 沈华伟, 程学启, 和蔡达生. 2024. 链内搜索: 通过搜索交互增强知识密集型任务的大语言模型. 载于 2024 年 ACM 网络会议论文集, WWW 2024, 新加坡, 2024 年 5 月 13-17 日, 第 1362-1373 页. ACM.

Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024. Corrective retrieval augmented generation. CoRR, abs/2401.15884.
闫诗琦, 顾佳辰, 朱云, 和凌振华. 2024. 纠正性检索增强生成. CoRR, abs/2401.15884.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. 2024. Qwen2.5 technical report. CoRR, abs/2412.15115.
杨安, 杨宝松, 张彬彦, 惠斌元, 郑波, 余博文, 李晨元, 刘大义恒, 黄飞, 魏浩然, 林欢, 杨建, 屠建宏, 张建伟, 杨建新, 杨佳茜, 周景仁, 林军阳, 党凯, 陆克明, 鲍克勤, 杨可欣, 余乐, 李梅, 薛明峰, 张培, 朱琴, 门瑞, 林润吉, 李天豪, 夏廷宇, 任行章, 任轩成, 樊洋, 苏阳, 张艺昌, 万宇, 刘玉琼, 崔泽宇, 张振如, 和邱子涵. 2024. Qwen2.5 技术报告. CoRR, abs/2412.15115.

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and
杨志林, 齐鹏, 张赛政, 约书亚·本吉奥, 威廉·W·科恩, 鲁斯兰·萨拉胡特丁诺夫, 和

Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 2369-2380. Association for Computational Linguistics.
Christopher D. Manning. 2018. Hotpotqa：多样化、可解释的多跳问题回答数据集。载于《2018 年自然语言处理实证方法会议论文集》，比利时布鲁塞尔，2018 年 10 月 31 日-11 月 4 日，第 2369-2380 页。计算语言学协会。

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023a. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
Shunyu Yao、Dian Yu、Jeffrey Zhao、Izhak Shafran、Tom Griffiths、Yuan Cao 和 Karthik Narasimhan. 2023a. 思想树：大型语言模型的深思熟虑的问题求解。载于《第 36 届神经信息处理系统年会：神经信息处理系统 2023 大会》，美国新奥尔良，2023 年 12 月 10-16 日。

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023b. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
Shunyu Yao、Jeffrey Zhao、Dian Yu、Nan Du、Izhak Shafran、Karthik R. Narasimhan 和 Yuan Cao. 2023b. ReAct：语言模型中推理与行动的协同。载于《第十一届学习表征国际会议》，卢旺达基加利，2023 年 5 月 1-5 日。OpenReview.net。

Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Raghavi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. 2024. Agent lumos: Unified and modular training for open-source language agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 12380-12403. Association for Computational Linguistics.
Da Yin、Faeze Brahman、Abhilasha Ravichander、Khyathi Raghavi Chandu、Kai-Wei Chang、Yejin Choi 和 Bill Yuchen Lin. 2024. Agent Lumos：开源语言智能体的统一和模块化训练。载于《计算语言学协会第 62 届年会论文集（第 1 卷：长论文）》，泰国曼谷，2024 年 8 月 11-16 日，第 12380-12403 页。计算语言学协会。

Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang, Shulin Cao, Shuxun Yang, Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang, Xixuan Song, Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang, Yueyan Li, Yushi Bai, Yuxiao Dong, Zehan Qi, Zhaoyu Wang, Zhen Yang, Zhengxiao Du, Zhenyu Hou, and Zihan Wang. 2024. Chatglm: A family of large language models from GLM-130B to GLM-4 all tools. CoRR, abs/2406.12793.
曾傲涵、徐斌、王博文、张晨辉、殷达、迭戈·罗哈斯、冯冠宇、赵翰林、赖汉宇、余浩、王洪宁、孙佳迪、张佳杰、程佳乐、桂佳怡、唐杰、张静、李娟子、赵磊、吴林东、钟鲁晨、刘明道、黄民列、张鹏、郑钦凯、卢锐、段帅琦、张舒丹、曹舒林、杨舒鑫、谭永霖、赵文义、刘小、夏小、张晓涵、顾小涛、吕欣、刘行汉、刘新宇、杨新悦、宋熙轩、张讯凯、安一帆、徐一帆、牛一林、杨元涛、李悦妍、白宇实、董跃晓、戚泽涵、王兆宇、杨震、杜政潇、侯振宇和王子涵. 2024. ChatGLM: 从 GLM-130B 到 GLM-4 的大语言模型家族. CoRR, abs/2406.12793.

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10-16, 2023.
郑连民, 蒋伟林, 盛颖, 庄思远, 吴樟浩, 庄永浩, 林梓, 李卓涵, 李大成, Eric P. Xing, 张浩, Joseph E. Gonzalez, 和 Ion Stoica. 2023. 使用 MT-Bench 和聊天机器人竞技场评判 LLM 作为裁判. 收录于神经信息处理系统进展 36: 2023 年神经信息处理系统年度会议, NeurIPS 2023, 美国新奥尔良, 2023 年 12 月 10-16 日.

Andrew Zhu, Alyssa Hwang, Liam Dugan, and Chris Callison-Burch. 2024. Fanoutqa: A multi-hop, multidocument question answering benchmark for large
朱安德鲁、黄艾丽莎、杜根利亚姆和卡利森-伯奇克里斯。2024. Fanoutqa：面向大型语言模型的多跳、多文档问答基准测试
language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024 - Short Papers, Bangkok, Thailand, August 11-16, 2024, pages 18-37. Association for Computational Linguistics.
语言模型。载《第 62 届计算语言学协会年会论文集》，ACL 2024 - 短文论文，泰国曼谷，2024 年 8 月 11-16 日，第 18-37 页。计算语言学协会。

Table 3: Closed-book performance of language models on multi-hop and single-hop benchmarks. These models perform better on single-hop benchmarks but score significantly lower on multi-hop benchmarks, highlighting the limitations of relying solely on parametric knowledge for these benchmarks.
表 3：语言模型在多跳和单跳基准测试中的封闭书本性能。这些模型在单跳基准测试中表现更好，但在多跳基准测试中得分显著较低，突显了仅依赖参数化知识的局限性。

Table 4: Performance comparison of models with and without strong reasoning capabilities. w/ reasoning consistently outperforms w/o reasoning across all benchmarks, demonstrating the effectiveness of our fine-tuning process, which enables ReaRAG-9B to inherit the strong reasoning abilities of LRM.
表 4：具有和不具有强推理能力的模型的性能比较。具有推理的模型在所有基准测试中始终优于无推理的模型，证明了我们微调过程的有效性，使 ReaRAG-9B 能够继承 LRM 的强大推理能力。

Algorithm 3 Data Construction to fine-tune ReaRAG w/o reasoning
Input: Seed Dataset \(\mathcal{D}_{\text {seed }}\), Large Language Model \(\mathcal{M}_{L L M}\), Instruction Prompt \(\mathcal{P}_{\text {ablat }}\)
Output: Dataset \(\mathcal{D}_{\text {ablat }}\)
    Initialize \(\mathcal{D}_{\text {ablat }} \leftarrow \emptyset\)
    for each \(\left(x_{i}, y_{i}\right.\), decomp \(\left._{i}\right) \sim \mathcal{D}_{\text {seed }}\) do
            \(\triangleright\) Sample question \(x_{i}\), ground truth answer \(y_{i}\) and golden decomposition decomp \({ }_{i}\)
        \(r_{i}^{\prime} \leftarrow \mathcal{M}_{\text {LLM }}\left(\left[\mathcal{P}_{\text {ablat }} \oplus x_{i} \oplus y_{i} \oplus\right.\right.\) decomp \(\left.\left._{i}\right]\right)\)
        \(\triangleright\) Generate response
        \(\mathcal{C}_{i}=\left[\left\{\tau_{t}, \alpha_{t}, o_{t}\right\}_{t=1}^{N}\right] \leftarrow \operatorname{parse}\left(r_{i}^{\prime}\right)\)
            \(\triangleright\) Parse a list of thought \(\tau_{t}\), action \(\alpha_{t}\) and observation \(o_{t}\) into reasoning chain \(\mathcal{C}_{i}\)
        Append \(\mathcal{C}_{i}\) to \(\mathcal{D}_{\text {ablat }}\)
    end for
    return \(\mathcal{D}_{\text {ablat }}\)

Your task is to solve a question answering task. To improve your solving accuracy, please conduct reasoning processes following this sequence: Thought, Action, Observation steps. Thought can reason about the current situation, and Action is in the form of a function. There are two available function types:
你的任务是解决一个问题回答任务。为了提高解决准确性，请按照以下顺序进行推理过程：思考、行动、观察步骤。思考可以推理当前情况，行动以函数形式呈现。有两种可用的函数类型：

(1) Search  (1) 搜索
{

"name": "search",
"description": "It can help you find useful information through the internet or local
    knowledge base. You can use this tool to access external knowledge.",
"parameters": {
    "type": "object",
    "properties": {
        "query": {
            "description": "what you want to search"
        }
    },
    "required": ["query"]
}
}

(2) Finish  (2) 完成
{

    "name": "finish",
    "description": "You can use this function to make a conclusion from the reasoning process
        and give the final answer. The reasoning process is completed after this 'finish'
        function is called",
    "parameters": {
        "type": "object",
        "properties": {
            "answer": {
                "description": "the final answer"
            }
        },
        "required": ["answer"]
    }
}

(1) Please follow the function calling format above strictly.
(1) 请严格遵循上面的函数调用格式。
(2) A set of ‘Thought’, ‘Action’, and ‘Observation’ is considered as one reasoning step. Add numbering after each
(2) 一组"思考"、"行动"和"观察"被视为一个推理步骤。在每个之后添加编号以指示推理步骤的顺序。
‘Thought’, ‘Action’, and ‘Observation’ to indicate the sequence of the reasoning steps.
"思考"、"行动"和"观察"来表示推理步骤的序列。
(3) Please give your ‘Thought’ first, then the ‘Action’, and finally the ‘Observation’, follow the format as shown in the in-context examples below.
(3) 请先给出你的"思考"，然后是"行动"，最后是"观察"，按照下面上下文示例中显示的格式进行。
(4) In your ‘Thought’, you should perform reflection when necessary to ensure the correctness of your reasoning process, such as: “Wait! Maybe I made some mistakes! I need to rethink from scratch”, “Alternatively, we can…”, “Hold on, let’s try another approach”, etc.
(4) 在你的"思考"中，必要时应进行反思，以确保推理过程的正确性，例如："等等！我可能犯了一些错误！我需要从头开始重新思考"、"另外，我们可以……"、"且慢，让我们尝试另一种方法"等。
(5) Give your ‘Action’ in the form of function call, as shown in in-context examples below.
(5) 按照下面上下文示例中所示，以函数调用的形式给出你的"动作"。
(6) You should not provide information based on your own knowledge, only use the information provided in the context.
(6) 你不应该基于自己的知识提供信息，只能使用上下文中提供的信息。

“There is no enough information from the previous steps. I need to plan my query again.”
前面的步骤中没有足够的信息。我需要重新规划我的查询。
“No context found from observation. Let me restart the reasoning process.”
未从观察中找到任何上下文。让我重新开始推理过程。
“Missing information. Let me restructure my query.”
缺少信息。让我重新构建我的查询。
“Wait! Maybe I made some mistakes! I need to rethink from scratch.”
等等！也许我犯了一些错误！我需要从头开始重新思考。
“I think I need to take a step back and reconsider my approach.”
我认为我需要退一步，重新考虑我的方法。
“I need to reevaluate my reasoning process. Let’s start over.”
我需要重新评估我的推理过程。让我们重新开始。
“I need to reflect on my reasoning. Let’s try a different approach.”
我需要反思我的推理过程。让我们尝试一种不同的方法。
{More examples of reflection text. Simplfied for readability}
更多反射文本示例。为了提高可读性而简化

{Some in-context examples}
{一些上下文示例}

Your task is to solve a question answering task. To improve your solving accuracy, please conduct reasoning process interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action are in the form of function, there are two types:
你的任务是解决问题回答任务。为了提高解决准确性，请进行推理过程，交替使用思考、行动和观察步骤。思考可以推理当前情况，行动采用函数形式，有两种类型：

Available Functions:
(1) Search
{
    "name": "search",
    "description": "It can help you find useful information through the internet or local
        knowledge base. You can use this tool to access external knowledge.",
    "parameters": {
        "type": "object",
        "properties": {
            "query": {
                "description": "what you want to search"
            }
        },
        "required": ["query"]
    }
}
(2) Finish
\{
    "name": "finish",
    "description": "You can use this function to make a conclusion from the reasoning process
        and give the final answer. The reasoning process is completed after this 'finish'
        function is called",
    "parameters": {
        "type": "object",
        "properties": {
            "answer": {
                "description": "the final answer"
            }
        },
        "required": ["answer"]
    }
}

Please follow the format strictly.
请严格遵循格式。

You are a QA assistant. Always return a short answer. Output ONLY the answer with no extra words.<luserl>
你是一个问答助手。始终返回简短的答案。仅输出答案，不加任何额外文字。<luserl>
You have conducted multiple searches to seek for informations to answer the Question.
你已进行多次搜索以寻找信息来回答问题。

{Previous search query and search results}
{先前的搜索查询和搜索结果}

Answer the Question based on the searches. Only output the final answer. Do NOT add any explanation, punctuation, or extra words.
根据搜索结果回答问题。仅输出最终答案。不要添加任何解释、标点符号或额外的文字。
Question: {question}  问题：{question}
Answer: <lassistantl>  答案：<lassistantl>