G-Designer: Architecting Multi-agent Communication
Topologies via Graph Neural Networks
G-Designer：通过图神经网络构建多智能体通信拓扑结构

While the academic community has widely recognized the success of single LLM-based agents in reasoning (Wei et al., 2022; Yao et al., 2023a; Besta et al., 2023) and planning (Huang et al., 2022; Sun et al., 2023b; Ruan et al., 2023), collaboration among multiple LLM-based agents has swiftly emerged as a powerful approach for integrating the specialized capabilities of different agents, even exceeding the performance of individual LLMs (Park et al., 2023; Chen et al., 2023a; Chan et al., 2023; Chen et al., 2023b; Cohen et al., 2023; Hua et al., 2023). A basic form of collaboration is majority voting (Chen et al., 2024a), where agents operate independently. However, more effective multi-agent collaboration should construct an interconnected system and iterative topology that encourages interdependent interactions and deliberate decision-making (Piatti et al., 2024; Chen et al., 2024a). Building on this insight, pioneering research has explored various multi-agent communication topologies, including: (1) Non-interactive, where agents operate independently without inter-agent communication, as employed in systems like LATM (Zhang et al., 2023b) and LLM-Debate (Du et al., 2023); (2) Chain, where agents are arranged in a sequential structure, each receiving the output from its predecessor and passing information to its successor, utilized by ChatDev (Qian et al., 2023), MetaGPT (Hong et al., 2023), and L2MAC (Holt et al., 2024); (3) Star, where a central administrative agent (often referred to as a commander, manager, teacher, etc.) directs subordinate agents, seen in AutoGen (Wu et al., 2023), SecurityBot (Yan et al., 2024), and MiniGrid (Zhou et al., 2023); (4) Tree, where a root or parent agent hierarchically manages multiple child agents, as in SoA (Ishibashi and Nishimura, 2024); and (5) Graph, encompassing complete graphs (Qian et al., 2024; Zhuge et al., 2024), layered graphs (Liu et al., 2023; Hao et al., 2023; Qian et al., 2024), and random graphs (Qian et al., 2024), among others.
尽管学术界普遍认可基于LLM的代理在推理（Wei 等人，2022；Yao 等人，2023a；Besta 等人，2023）和规划（Huang 等人，2022；Sun 等人，2023b；Ruan 等人，2023）方面的成功，但多个基于LLM的代理之间的协作迅速成为一种强大的方法，用于整合不同代理的专业能力，甚至超过了单个LLMs（Park 等人，2023；Chen 等人，2023a；Chan 等人，2023；Chen 等人，2023b；Cohen 等人，2023；Hua 等人，2023）的性能。协作的基本形式是多数投票（Chen 等人，2024a），其中代理独立操作。然而，更有效的多代理协作应构建一个相互连接的系统以及迭代拓扑，以鼓励相互依赖的交互和深思熟虑的决策（Piatti 等人，2024；Chen 等人，2024a）。 在此基础上，开创性研究探讨了各种多智能体通信拓扑结构，包括：（1）非交互式，智能体独立运行，无需智能体间通信，如 LATM（张等，2023b）和LLM-Debate（杜等，2023）所采用；（2）链式，智能体按顺序排列，每个智能体接收其前驱的输出并将信息传递给后继者，被 ChatDev（钱等，2023）、MetaGPT（洪等，2023）和 L2MAC（霍尔特等，2024）采用；（3）星形，中心管理智能体（通常称为指挥官、经理、教师等）指导下属智能体，如在 AutoGen（吴等，2023）、SecurityBot（严等，2024）和 MiniGrid（周等，2023）中看到；（4）树形，根或父智能体分层管理多个子智能体，如 SoA（石川和西村，2024）所示；（5）图形，包括完全图（钱等，2024；诸葛等，2024）、分层图（刘等，2023；郝等，2023；钱等，2024）和随机图（钱等，2024）等。

Graphs, as a fundamental data structure for organizing and representing relationships between entities (Bondy et al., 1976; Zhang and Chartrand, 2006), are widely adopted in the pre-LLM era as a powerful tool to facilitate effective communication in multi-agent reinforcement learning (MARL) (Pesce and Montana, 2023; Hu et al., 2024; Liu et al., 2022). With the rise of LLMs and the proliferation of LLM-based agents (Park et al., 2023; Chen et al., 2023a; Chan et al., 2023; Chen et al., 2023b; Cohen et al., 2023; Hua et al., 2023), researchers have similarly recognized that interactions among multiple agents can naturally be modeled from a graph-based perspective (Chen et al., 2023a; Zhuge et al., 2024; Qian et al., 2024; Liu et al., 2023). Early attempts are implicit, where ChatEval (Chan et al., 2023) and AutoGen (Wu et al., 2023) employ fixed graphs to facilitate information exchange among agents. Subsequent works, such as STOP (Zelikman et al., 2023) and DSPy (Khattab et al., 2023), further explore joint optimization of prompts and inference structures. More recent practices including ChatLLM (Hao et al., 2023), DyLAN (Liu et al., 2023), GPTSwarm (Zhuge et al., 2024), and MacNet (Qian et al., 2024), have explicitly represented the organization of multiple agents as a graph. Specifically, both ChatLLM (Hao et al., 2023) and DyLAN (Liu et al., 2023) utilize a multilayer perception (MLP)-like layered graph, while MacNet (Qian et al., 2024) systematically evaluates various predefined topologies. GPTSwarm (Zhuge et al., 2024) parameterizes and optimizes the fully-connected graph distribution. However, all these attempts, whether predefined static topologies or those iteratively optimized, remain input-independent. Consequently, they fail to be task-aware and adaptively design topologies that suit the complexity of the specific task.
图作为组织和表示实体之间关系的根本数据结构（Bondy 等，1976；张和查兰德，2006），在LLM之前的时代被广泛采用作为促进多智能体强化学习（MARL）中有效沟通的有力工具（Pesce 和 Montana，2023；胡等，2024；刘等，2022）。随着LLMs的兴起和基于LLM的智能体的激增（Park 等，2023；陈等，2023a；Chan 等，2023；陈等，2023b；Cohen 等，2023；华等，2023），研究人员同样认识到，多个智能体之间的交互可以从图的角度自然建模（陈等，2023a；诸葛等，2024；钱等，2024；刘等，2023）。早期的尝试是隐式的，其中 ChatEval（Chan 等，2023）和 AutoGen（吴等，2023）使用固定图来促进智能体之间的信息交换。后续工作，如 STOP（Zelikman 等，2023）和 DSPy（Khattab 等，2023），进一步探索了提示和推理结构的联合优化。 近期的研究实践包括 ChatLLM（郝等，2023 年）、DyLAN（刘等，2023 年）、GPTSwarm（诸葛等，2024 年）和 MacNet（钱等，2024 年），这些研究明确地将多个智能体的组织结构表示为图。具体来说，ChatLLM（郝等，2023 年）和 DyLAN（刘等，2023 年）利用类似于多层感知器（MLP）的多层图，而 MacNet（钱等，2024 年）系统地评估了各种预定义拓扑结构。GPTSwarm（诸葛等，2024 年）对全连接图分布进行参数化和优化。然而，所有这些尝试，无论是预定义的静态拓扑还是迭代优化的，都保持与输入无关。因此，它们无法感知任务并自适应地设计适合特定任务复杂性的拓扑结构。

We model the multi-agent system as a directed graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$, where $\mathcal{V}=\{v_{1},v_{2},\dots,v_{N}\}$ represents the set of nodes (with $N=|\mathcal{V}|$) and $\mathcal{E}$ denotes the set of edges. Each node $v_{i}\in\mathcal{V}$ corresponds to an agent, which can be formalized as:
我们将多智能体系统建模为一个有向图 $\mathcal{G}=(\mathcal{V},\mathcal{E})$ ，其中 $\mathcal{V}=\{v_{1},v_{2},\dots,v_{N}\}$ 代表节点集（带有 $N=|\mathcal{V}|$ ）， $\mathcal{E}$ 表示边集。每个节点 $v_{i}\in\mathcal{V}$ 对应一个智能体，可以形式化为：

where each agent $v_{i}$ is composed of four key elements: (1) $\texttt{Base}_{i}$, the language model instance powering $v_{i}$; (2) $\texttt{Role}_{i}$, the agent’s pre-assigned role or function; (3) $\texttt{State}_{i}$, representing the agent’s accumulated knowledge and interaction history; and (4) $\texttt{Plugin}_{i}$, a set of external tools or plugins available to $v_{i}$, such as web searchers (Ma et al., 2023), code compilers (Richards and et al., 2023; Wu et al., 2023; Hong et al., 2023; Bouzenia et al., 2024; Ishibashi and Nishimura, 2024), or file readers (Zhuge et al., 2024; Richards and et al., 2023). Each LLM-based agent $v_{i}$ receives prompt $\mathcal{P}$ and generates response $\mathcal{R}_{i}$:
每个智能体 $v_{i}$ 由四个关键元素组成：（1） $\texttt{Base}_{i}$ ，为 $v_{i}$ 提供动力的语言模型实例；（2） $\texttt{Role}_{i}$ ，智能体的预分配角色或功能；（3） $\texttt{State}_{i}$ ，代表智能体的累积知识和交互历史；（4） $\texttt{Plugin}_{i}$ ，一组可供 $v_{i}$ 使用的工具或插件，例如网络搜索器（Ma 等，2023 年）、代码编译器（Richards 等人，2023 年；Wu 等人，2023 年；Hong 等人，2023 年；Bouzenia 等人，2024 年；Ishibashi 和 Nishimura，2024 年）或文件阅读器（Zhuge 等人，2024 年；Richards 等人，2023 年）。每个基于LLM的智能体 $v_{i}$ 接收提示 $\mathcal{P}$ 并生成响应 $\mathcal{R}_{i}$ ：

where $\mathcal{P}_{\text{sys}}=\{\texttt{Role}_{i},\texttt{State}_{i}\}$ represents the system prompt encompassing its role and state, and $\mathcal{P}_{\text{usr}}$ denotes the user prompt, which possibly includes the given tasks, responses/instructions from other agents and externally retrieved knowledge.
$\mathcal{P}_{\text{sys}}=\{\texttt{Role}_{i},\texttt{State}_{i}\}$ 代表包含其角色和状态的系统提示， $\mathcal{P}_{\text{usr}}$ 表示用户提示，可能包括分配的任务、其他代理的响应/指令以及从外部检索到的知识。

The connectivity of $\mathcal{G}$ can also be characterized by a (non-symmetric) adjacency matrix $\mathbf{A}\in\{0,1\}^{N\times N}$, where $\mathbf{A}[i,j]=1$ if $e_{ij}=(v_{i},v_{j})\in\mathcal{E}$, otherwise $0$. Each edge $e_{ij}\in\mathcal{E}$ represents the flow of information from agent $v_{i}$ to agent $v_{j}$.
$\mathcal{G}$ 的连通性也可以通过一个（非对称的）邻接矩阵 $\mathbf{A}\in\{0,1\}^{N\times N}$ 来描述，其中 $\mathbf{A}[i,j]=1$ 当且仅当 $e_{ij}=(v_{i},v_{j})\in\mathcal{E}$ ，否则 $0$ 。每条边 $e_{ij}\in\mathcal{E}$ 代表从代理 $v_{i}$ 到代理 $v_{j}$ 的信息流。

Given a query/problem $\mathcal{Q}$, the multi-agent system engages in $K$ rounds of interactive utterances, which collaboratively drive the agents toward producing the final solution $a^{(K)}$ based on their cumulative dialogue exchanges. At the beginning of the $t$-th dialogue round, a mapping function $\phi$ is applied to determine the execution index for each agent:
给定一个查询/问题 $\mathcal{Q}$ ，多智能体系统进行 $K$ 轮交互对话，通过协同推动智能体根据其累积的对话交流产生最终解决方案 $a^{(K)}$ 。在第 $t$ 轮对话开始时，应用映射函数 $\phi$ 确定每个智能体的执行索引：

where $\sigma$ is the execution sequence of agents, $\mathcal{N}_{\text{in}}(v_{\sigma(j)})$ denotes the in-neighborhood of $v_{\sigma(j)}$, and the constraint ensures that an agent $v_{\sigma(i)}$ can only execute after any agent $v_{\sigma(j)}$ from which it receives information. Once the execution order is determined, each agent proceeds to perform input-output operations sequentially:
$\sigma$ 表示代理的执行序列， $\mathcal{N}_{\text{in}}(v_{\sigma(j)})$ 表示 $v_{\sigma(j)}$ 的邻域，约束条件确保代理 $v_{\sigma(i)}$ 只能在接收到信息后的任何代理 $v_{\sigma(j)}$ 执行。一旦确定执行顺序，每个代理将依次执行输入输出操作：

where $\mathcal{R}_{i}^{(t)}$ represents the output of agent $v_{i}$, which could be a rationale, an answer, or a partial solution, depending on the specific context. The output $\mathcal{R}_{i}^{(t)}$ is generated based on the system prompt $\mathcal{P}_{\text{sys}}^{(t)}$ and the context prompt, consisting of the query $\mathcal{Q}$ and messages from other agents. At the end of each dialogue round, a certain aggregation function is adopted to generate the answer/solution $a^{(t)}$ based on the dialogue history:
$\mathcal{R}_{i}^{(t)}$ 代表智能体 $v_{i}$ 的输出，可能是理由、答案或部分解决方案，具体取决于特定上下文。输出 $\mathcal{R}_{i}^{(t)}$ 基于系统提示 $\mathcal{P}_{\text{sys}}^{(t)}$ 和上下文提示（包括查询 $\mathcal{Q}$ 和其他智能体的消息）生成。在每个对话回合结束时，采用某种聚合函数根据对话历史生成答案/解决方案 $a^{(t)}$ 。

The implementation of the $\operatorname{Aggregate}$ function is flexible, with possible options including majority voting (Chen et al., 2024a; Zhuge et al., 2024; Li et al., 2024), aggregating all agents’ responses and delegating one agent to provide the final answer (Wu et al., 2023; Jiang et al., 2023; Liu et al., 2023; Zhang et al., 2024), or simply using the output of the last agent $\mathcal{R}^{(t)}_{\sigma_{N}}$ (Qian et al., 2024). Through $K$ rounds of utterances, either predefined (Qian et al., 2024) or determined by an early-stopping mechanism (Liu et al., 2023), the overall system $\mathcal{G}$ produces the final answer $a^{(K)}$ in response to $\mathcal{Q}$.
函数 $\operatorname{Aggregate}$ 的实现具有灵活性，包括多数投票（Chen 等，2024a；Zhuge 等，2024；Li 等，2024）、汇总所有代理的响应并委托一个代理提供最终答案（Wu 等，2023；Jiang 等，2023；Liu 等，2023；Zhang 等，2024），或简单地使用最后一个代理 $\mathcal{R}^{(t)}_{\sigma_{N}}$ 的输出（Qian 等，2024）。通过 $K$ 轮的对话，可以是预定义的（Qian 等，2024）或由早期停止机制决定（Liu 等，2023），整体系统 $\mathcal{G}$ 针对 $\mathcal{Q}$ 产生最终答案 $a^{(K)}$ 。

We give the formal definition of MACP Protocol as follows:
我们给出 MACP 协议的正式定义如下：

Given an input query $\mathcal{Q}$ and a set of LLM-agents $\mathcal{V}$, G-Designer aims to design a task-adaptive and effective communication topology $\mathcal{G}_{com}$. We begin by assigning each agent a unique role and profile, as previous research (Wang et al., 2023a) has shown that assigning distinct personas or roles to LLM-based agents can enhance cognitive synergy. Based on these roles, different external tools are allocated to the agents (e.g., Mathematica for a math analyst, Python compiler for a programmer). Thus, we successfully initialize each agent $v_{i}$ as $\{\texttt{Base}_{i},\texttt{Role}_{i},\texttt{State}_{i},\texttt{Plugin}_{i}\}$, as defined in Equation 1.
给定输入查询 $\mathcal{Q}$ 和一组LLM-代理 $\mathcal{V}$ ，G-Designer 旨在设计一个任务自适应和有效的通信拓扑 $\mathcal{G}_{com}$ 。我们首先为每个代理分配一个独特的角色和配置文件，因为先前的研究（王等，2023a）表明，将不同的角色或个性分配给LLM-基于的代理可以增强认知协同。基于这些角色，不同的外部工具被分配给代理（例如，Mathematica 用于数学分析师，Python 编译器用于程序员）。因此，我们成功地将每个代理 $v_{i}$ 初始化为 $\{\texttt{Base}_{i},\texttt{Role}_{i},\texttt{State}_{i},\texttt{Plugin}_{i}\}$ ，如方程 1 所定义。

We proceed to construct a structured multi-agent network as input to G-Designer, represented as $\mathcal{G}=(\mathbf{X}_{agent},\mathbf{A})$, where $\mathbf{X}_{agent}\in\mathbb{R}^{N\times D}$ is the node (agent) feature matrix and $\mathbf{A}\in\mathbb{R}^{N\times N}$ represents the connectivity matrix. For the feature matrix $\mathbf{X}_{agent}$, we employ a node encoder to transform each agent’s unique profile into a fixed-length embedding representation:
我们继续构建一个结构化多智能体网络，作为 G-Designer 的输入，表示为 $\mathcal{G}=(\mathbf{X}_{agent},\mathbf{A})$ ，其中 $\mathbf{X}_{agent}\in\mathbb{R}^{N\times D}$ 是节点（智能体）特征矩阵， $\mathbf{A}\in\mathbb{R}^{N\times N}$ 表示连接矩阵。对于特征矩阵 $\mathbf{X}_{agent}$ ，我们使用节点编码器将每个智能体的独特配置文件转换为固定长度的嵌入表示：

where $\mathcal{T}(\cdot)$ extracts the textual description of the agent’s LLM backbone and its assigned plugins, and $\operatorname{NodeEncoder}$ can be realized using small and lightweight text embedding models such as SentenceBERT (Reimers, 2019) or MiniLM (Wang et al., 2020). After encoding the individual agents, we aim to ensure that the multi-agent network incorporates information related to the query $\mathcal{Q}$, as this query-dependent approach enables G-Designer to be task-aware and adaptive. To this end, we introduce an additional task-specific virtual global node $v_{task}$, which is bidirectionally connected to all agent nodes, enabling a global ”storage sink” and facilitating smoother information flow among agents (Shirzad et al., 2023; Tan et al., 2023; Rosenbluth et al., 2024). This task node is encoded by the $\operatorname{NodeEncoder}$ as follows: $\mathbf{x}_{task}\leftarrow\operatorname{NodeEncoder}(\mathcal{Q})$.
$\mathcal{T}(\cdot)$ 提取代理的 LLM 背骨及其分配的插件文本描述，而 $\operatorname{NodeEncoder}$ 可以通过使用小型轻量级文本嵌入模型如 SentenceBERT（Reimers，2019）或 MiniLM（Wang 等人，2020）来实现。在编码单个代理之后，我们旨在确保多代理网络包含与查询 $\mathcal{Q}$ 相关的信息，因为这种基于查询的方法使 G-Designer 能够感知任务并具有适应性。为此，我们引入了一个额外的特定任务虚拟全局节点 $v_{task}$ ，该节点双向连接到所有代理节点，实现全局“存储汇”并促进代理之间更流畅的信息流动（Shirzad 等人，2023；Tan 等人，2023；Rosenbluth 等人，2024）。此任务节点由 $\operatorname{NodeEncoder}$ 按以下方式编码： $\mathbf{x}_{task}\leftarrow\operatorname{NodeEncoder}(\mathcal{Q})$ 。

After obtaining the agent node features $\mathbf{X}_{agent}=[\mathbf{x}_{1},\mathbf{x}_{2},\dots,\mathbf{x}_{N}]^{\top}$ and the task-specific embedding $\mathbf{x}_{task}$, we provide a simple anchor topology $\mathbf{A}_{anchor}\in\{0,1\}^{N\times N}$, which serves as a starting point for G-Designer’s topology design process. For instance, given a code generation task with three agents: manager/programmer/code reviewer, the anchor topology could be configured as a chain structure, i.e., “manager $\rightarrow$ programmer $\rightarrow$ reviewer”, reflecting the typical workflow of code completion. The anchor topology, being either user-defined or automatically generated by LLMs, is often simple and sub-optimal. However, it provides a foundational reference and prior knowledge for G-Designer’s subsequent optimization process. We incorporate the task-specific vertex $v_{task}$ and its corresponding edges and obtain $\tilde{\mathbf{A}}_{anchor}\in\{0,1\}^{(N+1)\times(N+1)}$. Consequently, we establish a task-specific multi-agent network $\tilde{\mathcal{G}}$:
在获取代理节点特征 $\mathbf{X}_{agent}=[\mathbf{x}_{1},\mathbf{x}_{2},\dots,\mathbf{x}_{N}]^{\top}$ 和任务特定嵌入 $\mathbf{x}_{task}$ 后，我们提供了一个简单的锚拓扑 $\mathbf{A}_{anchor}\in\{0,1\}^{N\times N}$ ，这作为 G-Designer 拓扑设计过程的起点。例如，给定一个有三个代理的代码生成任务：经理/程序员/代码审查员，锚拓扑可以配置为链结构，即“经理 $\rightarrow$ 程序员 $\rightarrow$ 审查员”，反映代码补全的典型工作流程。锚拓扑，无论是用户定义的还是由 LLMs 自动生成的，通常很简单且次优。然而，它为 G-Designer 的后续优化过程提供了一个基础参考和先验知识。我们纳入任务特定的顶点 $v_{task}$ 及其对应的边，并得到 $\tilde{\mathbf{A}}_{anchor}\in\{0,1\}^{(N+1)\times(N+1)}$ 。因此，我们建立了一个任务特定的多代理网络 $\tilde{\mathcal{G}}$ 。

where $\scriptsize\begin{bmatrix}\mathbf{X}_{agent}\\ \mathbf{x}_{task}^{\top}\end{bmatrix}$ can also be jointly denoted as $\tilde{\mathbf{X}}$.
$\scriptsize\begin{bmatrix}\mathbf{X}_{agent}\\ \mathbf{x}_{task}^{\top}\end{bmatrix}$ 也可以共同表示为 $\tilde{\mathbf{X}}$ 。

Building upon the task-specific multi-agent network $\tilde{\mathcal{G}}$, G-Designer seeks to establish a more fine-grained and precise communication topology $\mathcal{G}_{com}$. Drawing inspiration from the variational graph auto-encoder (VGAE) framework (Kipf and Welling, 2016; Zhao and Zhang, 2024), G-Designer employs a VGAE-based encoder-decoder $f_{v}$ to generate the multi-agent interaction topology, which is formulated as:
基于特定任务的多元代理网络 $\tilde{\mathcal{G}}$ ，G-Designer 旨在建立一个更精细和精确的通信拓扑 $\mathcal{G}_{com}$ 。从变分图自动编码器（VGAE）框架（Kipf 和 Welling，2016；Zhao 和 Zhang，2024）中汲取灵感，G-Designer 采用基于 VGAE 的编码器-解码器 $f_{v}$ 来生成多元代理交互拓扑，其公式为：

where $f_{v}$ is the encoder-decoder architecture with parameters $\Theta_{v}$, $q(\cdot)$ is the encoder module, $p(\cdot)$ is the decoder module. The encoder utilizes posterior probabilities to encode the node embeddings into low-dimensional latent vector representations $\mathbf{H}_{agent}$, which can be formulated as:
$f_{v}$ 是具有参数 $\Theta_{v}$ 的编码器-解码器架构， $q(\cdot)$ 是编码器模块， $p(\cdot)$ 是解码器模块。编码器利用后验概率将节点嵌入编码成低维潜在向量表示 $\mathbf{H}_{agent}$ ，可以表示为：

where $\mu=\operatorname{GNN}_{\mu}(\tilde{\mathbf{X}},\tilde{\mathbf{A}}_{anchor};\Theta_{\mu})$ is the matrix of mean vectors $\mu_{i}$; similarly $\log(\sigma)=\operatorname{GNN}_{\sigma}(\tilde{\mathbf{X}},\tilde{\mathbf{A}}_{anchor};\Theta_{\sigma})$. The choice of GNN backbone can be customized as needed; here, we utilize a simple two-layer GCN (Kipf and Welling, 2017). $\mathbf{h}_{i}$, $\boldsymbol{\mu}_{i}$, and $\boldsymbol{\sigma}_{i}$ denote the $i$-th column of $\mathbf{H}$, $\boldsymbol{\mu}$, and $\boldsymbol{\sigma}$, respectively. The encoder $q(\cdot)$ is parameterized by $\Theta_{e}=\{\Theta_{\mu},\Theta_{\sigma}\}$. Following the encoding phase, the decoder employs the latent representations to generate a comprehensive blueprint for multi-agent communication. More specifically, the decoder $q(\cdot)=q_{c}\circ q_{s}$ first constructs a parameterized, sketched graph $\mathbf{S}$, which is then refined into the final multi-agent communication topology:
$\mu=\operatorname{GNN}_{\mu}(\tilde{\mathbf{X}},\tilde{\mathbf{A}}_{anchor};\Theta_{\mu})$ 是均值向量矩阵 $\mu_{i}$ ；同样地， $\log(\sigma)=\operatorname{GNN}_{\sigma}(\tilde{\mathbf{X}},\tilde{\mathbf{A}}_{anchor};\Theta_{\sigma})$ 。GNN 骨干网络的选择可以根据需要定制；在这里，我们使用了一个简单的两层 GCN（Kipf 和 Welling，2017）。 $\mathbf{h}_{i}$ 、 $\boldsymbol{\mu}_{i}$ 和 $\boldsymbol{\sigma}_{i}$ 分别表示 $i$ 、 $\mathbf{H}$ 和 $\boldsymbol{\mu}$ 的第 $\boldsymbol{\sigma}$ 列。编码器 $q(\cdot)$ 由 $\Theta_{e}=\{\Theta_{\mu},\Theta_{\sigma}\}$ 参数化。在编码阶段之后，解码器使用潜在表示来生成多智能体通信的全面蓝图。更具体地说，解码器 $q(\cdot)=q_{c}\circ q_{s}$ 首先构建一个参数化的草图图 $\mathbf{S}$ ，然后将其细化成最终的多智能体通信拓扑结构：

At the first step, $p_{s}(\cdot)$ constructs the fully-connected sketched adjacency matrix $\mathbf{S}$ from the latent representation $\mathbf{H}_{agent}$:
在第一步， $p_{s}(\cdot)$ 从潜在表示 $\mathbf{H}_{agent}$ 构建了全连接的近似邻接矩阵 $\mathbf{S}$ ：

whose detailed derivation is as follows:
其详细推导如下：

where $\varpi=\operatorname{FFN}_{d}([\mathbf{h}_{i},\mathbf{h}_{j},\mathbf{h}_{task}])$ with $\operatorname{FFN}_{d}$ parameterized by $\Theta_{d}$, $\epsilon\sim\operatorname{Uniform}(0,1)$, and $\tau$ denotes the temperature coefficient. When $\tau$ approaches zero, Equation 13 essentially return the Bernouli sampling result for $\mathbf{S}_{ij}$. The resulting matrix $\mathbf{S}\in[0,1]^{N\times N}$ represents a densely-connected, non-negative graph distribution, indicating an overly complex and resource-intensive pair-wise communication structure, which is not yet suitable for guiding multi-agent collaboration. To align with G-Designer’s objectives of task adaptiveness and minimizing costs, we apply a refinement decoder $p_{c}(\cdot)$ to refine the sketched $\mathbf{S}$ into a compact, sparse, and highly informative communication graph, instantiated by a regularization objective:
在由 $\Theta_{d}$ 、 $\epsilon\sim\operatorname{Uniform}(0,1)$ 和 $\tau$ 参数化的 $\varpi=\operatorname{FFN}_{d}([\mathbf{h}_{i},\mathbf{h}_{j},\mathbf{h}_{task}])$ 中， $\epsilon\sim\operatorname{Uniform}(0,1)$ 表示温度系数。当 $\tau$ 趋近于零时，方程 13 本质上返回 $\mathbf{S}_{ij}$ 的伯努利采样结果。得到的矩阵 $\mathbf{S}\in[0,1]^{N\times N}$ 代表一个密集连接、非负图分布，表明一个过于复杂且资源密集的成对通信结构，目前还不适合指导多智能体协作。为了与 G-Designer 的任务适应性和最小化成本的目标相一致，我们应用一个细化解码器 $p_{c}(\cdot)$ 将草图 $\mathbf{S}$ 细化成一个紧凑、稀疏且高度信息化的通信图，通过正则化目标实现。

where $\mathbf{Z}\in\mathbb{R}^{N\times r}$ is the top-$r$ columns of left singular matrix $\mathbf{S}$, $\zeta$ is a coefficient hyperparameter, $\mathbf{W}\in\mathbb{R}^{r\times r}$ is an optimizable weight matrix, $||\cdot||_{F}$ denotes the Frobenius norm and $||\mathbf{W}||_{*}=\sum_{i}\lambda_{i}$ where $\lambda_{i}$ is the $i$-th singular value of $\mathbf{W}$. $\tilde{\mathbf{S}}\in\mathbb{R}^{N\times N}$ is the desired sparse topology, which is decomposed as $\mathbf{Z}\mathbf{W}\mathbf{Z}^{\top}$. In Equation 14, the first and second terms are jointly denoted as anchor regularization, which encourage the learned $\tilde{\mathbf{S}}$ to maintain similarity with both the original $\mathbf{S}$ and the anchor topology. The third term, denoted as sparsity regularization, though appearing to minimize the nuclear norm of $\mathbf{W}$, essentially sparsifies $\tilde{\mathbf{S}}$, since $||\tilde{\mathbf{S}}||_{*}=||\mathbf{W}||_{*}$ holds due to $\mathbf{Z}^{\top}\mathbf{Z}=\mathbbm{I}_{r\times r}$. Therefore, Equation 14 achieves two key goals: (1) producing a sparse, refined communication topology, and (2) constraining the design to remain grounded in practical intuition. The resulting communication design can be represented as follows:
$\mathbf{Z}\in\mathbb{R}^{N\times r}$ 是左奇异矩阵 $\mathbf{S}$ 的前 $r$ 列， $\zeta$ 是系数超参数， $\mathbf{W}\in\mathbb{R}^{r\times r}$ 是可优化的权重矩阵， $||\cdot||_{F}$ 表示 Frobenius 范数， $||\mathbf{W}||_{*}=\sum_{i}\lambda_{i}$ 中 $\lambda_{i}$ 是 $\mathbf{W}$ 的第 $i$ 个奇异值。 $\tilde{\mathbf{S}}\in\mathbb{R}^{N\times N}$ 是期望的稀疏拓扑，它被分解为 $\mathbf{Z}\mathbf{W}\mathbf{Z}^{\top}$ 。在方程 14 中，第一项和第二项共同表示锚正则化，它鼓励学习到的 $\tilde{\mathbf{S}}$ 与原始 $\mathbf{S}$ 和锚拓扑保持相似性。第三项，表示为稀疏正则化，虽然看起来是使 $\mathbf{W}$ 的核范数最小化，但实际上由于 $\mathbf{Z}^{\top}\mathbf{Z}=\mathbbm{I}_{r\times r}$ 的原因，本质上使 $\tilde{\mathbf{S}}$ 稀疏化。因此，方程 14 实现了两个关键目标：（1）生成一个稀疏、精细的通信拓扑，以及（2）将设计约束在实用直觉的基础上。结果通信设计可以表示如下：

At this stage, we have successfully distilled a lightweight and informative collaboration network $\mathcal{G}_{com}$ from the roughly constructed task-specific multi-agent network $\tilde{\mathcal{G}}$, which is now ready to guide inter-agent message passing in the following process.
在这个阶段，我们已成功从大致构建的任务特定多智能体网络 $\tilde{\mathcal{G}}$ 中提取出一个轻量级且信息丰富的协作网络 $\mathcal{G}_{com}$ ，该网络现在可以指导后续过程中的智能体间消息传递。

Upon obtaining $\mathcal{G}_{com}$, the multi-agent utterances and dialogues can proceed as usual using $\mathcal{G}_{com}$, as detailed in Section 3.2. After $K$ rounds of interaction, the agents converge to a final solution $a^{(K)}=\mathcal{G}_{com}(\mathcal{Q})$. We then give the following optimization objective:
在获得 $\mathcal{G}_{com}$ 后，多智能体的话语和对话可以像往常一样使用 $\mathcal{G}_{com}$ 进行，具体内容见第 3.2 节。经过 $K$ 轮交互后，智能体收敛到最终解 $a^{(K)}=\mathcal{G}_{com}(\mathcal{Q})$ 。然后我们给出以下优化目标：

where $\Theta_{e}$ and $\Theta_{d}$ are the parameters of the encoder $q(\cdot)$ and decoder $p(\cdot)$, respectively, $\Omega$ is the parameter space and $\mathbb{E}(\cdot)$ denotes the mathematical expectation. Equation 16 aims to maximize the utility of the generated solution, but it is inherently intractable and non-differentiable, as $u(\cdot)$ often depends on external API calls (Li et al., 2023b; Hendrycks et al., 2021). To address this, following standard approaches in multi-agent structure design (Zhuge et al., 2024; Zhang et al., 2024), we apply policy gradient (Williams, 1992) to approximate and optimize Equation 16:
$\Theta_{e}$ 和 $\Theta_{d}$ 分别是编码器 $q(\cdot)$ 和解码器 $p(\cdot)$ 的参数， $\Omega$ 表示参数空间， $\mathbb{E}(\cdot)$ 表示数学期望。方程 16 旨在最大化生成解的效用，但它本质上难以处理且不可微，因为 $u(\cdot)$ 通常依赖于外部 API 调用（Li 等，2023b；Hendrycks 等，2021）。为了解决这个问题，我们遵循多智能体结构设计中的标准方法（Zhuge 等，2024；Zhang 等，2024），应用策略梯度（Williams，1992）来近似和优化方程 16：

where $\mathbf{\Theta}=\{\Theta_{e},\Theta_{d}\}$, $\{\mathcal{G}_{k}\}_{m=1}^{M}$ are indepently samples from $\mathcal{G}_{com}$, and $\{a_{m}^{(K)}\}_{m=1}^{M}$ are the corresponding output. $P(\mathcal{G}_{k})$ calculates the probability of $\mathcal{G}_{k}$ being sampled, which can be expressed as $P(\mathcal{G}_{k})=\prod_{i=1}^{N}\prod_{j=1}^{N}\tilde{\mathbf{S}}_{ij}$. Through iterative optimization guided by Equations 14 and 16 over a limited set of queries as the “training set”, G-Designer efficiently develops task-awareness and the capability to strategically design the agent network, achieving truly task-customized multi-agent topology design.
$\mathbf{\Theta}=\{\Theta_{e},\Theta_{d}\}$ 和 $\{\mathcal{G}_{k}\}_{m=1}^{M}$ 是从 $\mathcal{G}_{com}$ 中独立抽取的样本， $\{a_{m}^{(K)}\}_{m=1}^{M}$ 是相应的输出。 $P(\mathcal{G}_{k})$ 计算被抽取的 $\mathcal{G}_{k}$ 的概率，可以表示为 $P(\mathcal{G}_{k})=\prod_{i=1}^{N}\prod_{j=1}^{N}\tilde{\mathbf{S}}_{ij}$ 。通过在有限的查询集上，由方程 14 和 16 引导的迭代优化，作为“训练集”，G-Designer 有效地发展了任务感知能力和战略性地设计代理网络的能力，实现了真正任务定制的多代理拓扑设计。

To assess the effectiveness of G-Designer in designing powerful LLM-MA topologies, we conducted evaluations using five instances of gpt-4, with results outlined in Table 1. We can draw two key observations (Obs.):
为了评估 G-Designer 在设计强大的LLM-MA 拓扑结构中的有效性，我们使用 gpt-4 的五个实例进行了评估，结果如表 1 所示。我们可以得出两个关键观察（观察）：

Task-adaptive multi-agent network design not only improves task performance but also regulates the complexity of the topology according to the task’s difficulty. A key benefit of this adaptivity is that it prevents the use of overly complex structures for simple tasks, thus minimizing unnecessary communication costs—in the case of LLM-MA, reducing token consumption. Figure 4 visualizes the different topologies designed by G-Designer for varying query difficulties on the HumanEval and GSM8K benchmarks, while Figure 5 compares the token consumption of G-Designer against various baselines. We summarize several interesting observations:
任务自适应多智能体网络设计不仅提高了任务性能，还能根据任务的难度调节拓扑结构的复杂性。这种自适应的关键好处是防止在简单任务中使用过于复杂的结构，从而最小化不必要的通信成本——在LLM-MA 的情况下，减少令牌消耗。图 4 展示了 G-Designer 为 HumanEval 和 GSM8K 基准测试中不同查询难度设计的不同拓扑结构，而图 5 比较了 G-Designer 与各种基线的令牌消耗。我们总结了几个有趣的观察结果：

In this section, we compare the robustness of different topology designs when subjected to adversarial attacks. Following (Zhuge et al., 2024), we simulate a system prompt attack on one of the five agents, with the results shown in Figure 6. We observe the following:
在本节中，我们比较了不同拓扑结构设计在遭受对抗攻击时的鲁棒性。遵循（Zhuge 等人，2024 年）的方法，我们对五个智能体中的一个进行了系统提示攻击模拟，结果如图 6 所示。我们观察到以下情况：