Compare commits
3 Commits
7b13c1278f
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
 ShiyuZhang
|
2897b32d0d | ||
|
ShiyuZhang
|
718e371eb5 | ||
|
ShiyuZhang
|
5f469a387a |
@@ -30,7 +30,7 @@ $$
|
|||||||
其中：
|
其中：
|
||||||
|
|
||||||
- $x_v$：节点能力和资源，例如是否能推理、拆分任务、调用工具或查询数据库；
|
- $x_v$：节点能力和资源，例如是否能推理、拆分任务、调用工具或查询数据库；
|
||||||
- $q_v$：节点当前内部状态，例如空闲、思考、等待或发送；
|
- $q_v$：节点当前的资源状态，例如空闲、忙碌、满载或失败。它是节点级状态，不表示某一条消息处于哪个业务处理阶段；
|
||||||
- $G_v$：节点周围的局部拓扑，例如邻居、连接数量、距离和邻居负载；
|
- $G_v$：节点周围的局部拓扑，例如邻居、连接数量、距离和邻居负载；
|
||||||
- $c_v$：当前任务上下文，例如任务类型、消息类型和任务阶段。
|
- $c_v$：当前任务上下文，例如任务类型、消息类型和任务阶段。
|
||||||
|
|
||||||
@@ -52,7 +52,7 @@ max_concurrency = 4
|
|||||||
消息是网络中实际传输的对象，可以抽象为：
|
消息是网络中实际传输的对象，可以抽象为：
|
||||||
|
|
||||||
$$
|
$$
|
||||||
m=(message\_id,task\_id,parent\_message\_id,source,destination,message\_type,message\_size,priority,path,h,\ldots)
|
m=(message\_id,task\_id,parent\_message\_id,correlation\_id,attempt\_id,source,destination,next\_hop,message\_type,message\_size,priority,path,h,\ldots)
|
||||||
$$
|
$$
|
||||||
|
|
||||||
其中：
|
其中：
|
||||||
@@ -60,29 +60,37 @@ $$
|
|||||||
- `message_id`：当前消息的唯一标识，用于追踪消息从产生到完成、失败或重试的完整生命周期；
|
- `message_id`：当前消息的唯一标识，用于追踪消息从产生到完成、失败或重试的完整生命周期；
|
||||||
- `task_id`：消息所属任务的唯一标识，用于关联同一任务产生的多条消息；
|
- `task_id`：消息所属任务的唯一标识，用于关联同一任务产生的多条消息；
|
||||||
- `parent_message_id`：产生当前消息的上游消息标识。对于任务拆分、工具调用和重试等场景，可据此恢复消息之间的触发关系；
|
- `parent_message_id`：产生当前消息的上游消息标识。对于任务拆分、工具调用和重试等场景，可据此恢复消息之间的触发关系；
|
||||||
|
- `correlation_id`：请求与响应之间的关联标识。工具响应、协作结果或错误消息使用它匹配对应的请求；
|
||||||
|
- `attempt_id`：一次具体发送尝试的标识。相同逻辑请求的重试可以拥有不同的 `attempt_id`；
|
||||||
- `source`：消息的逻辑发送端，即消息产生的节点；
|
- `source`：消息的逻辑发送端，即消息产生的节点；
|
||||||
- `destination`：消息的逻辑接收端，即本次消息最终要到达的节点。二者表示端到端关系，不表示多跳传播中的每一条链路；
|
- `destination`：消息的逻辑接收端，即本次消息最终要到达的节点。二者表示端到端关系，不表示多跳传播中的每一条链路；
|
||||||
|
- `next_hop`：本次传输实际要到达的下一跳节点。它可以等于 `destination`，也可以是通往 `destination` 的中间转发节点；
|
||||||
- `message_type`：消息的业务类型，决定节点收到消息后的处理方式以及可能产生的后续消息；
|
- `message_type`：消息的业务类型，决定节点收到消息后的处理方式以及可能产生的后续消息；
|
||||||
- `message_size`（或 $b$）：消息大小，统一以 Byte 计，用于计算链路和节点的字节流量；
|
- `message_size`（或 $b$）：消息大小，统一以 Byte 计，用于计算链路和节点的字节流量；
|
||||||
- `priority`：消息优先级，用于消息队列中的调度。其取值范围和相同优先级下的排序规则由系统配置决定；
|
- `priority`：消息优先级，用于消息队列中的调度。其取值范围和相同优先级下的排序规则由系统配置决定；
|
||||||
- `path`：消息计划经过的节点序列，记为 $(v_0,v_1,\ldots,v_k)$，其中通常有 $v_0=source$、$v_k=destination$。如果采用逐跳路由，`path` 可以只保存已经确定的部分，后续节点在转发时再补充；
|
- `path`：消息已经确认或实际经过的节点序列，记为 $(v_0,v_1,\ldots,v_h)$；使用预先计算路由时也可以保存完整计划路径。通常有 $v_0=source$，到达最终目标时最后一个节点为 `destination`；
|
||||||
- $h$：消息已经完成的跳数，取非负整数，表示消息当前位于 $v_h$，因此初始位于 $source=v_0$ 时 $h=0$，到达 $v_1$ 后 $h=1$。它不是“路径中的第几个节点”的自然序号，避免与从 1 开始的节点计数混淆；
|
- $h$：消息已经完成的跳数，取非负整数，表示消息当前位于已确认路径的 $v_h$，因此初始位于 $source=v_0$ 时 $h=0$，每成功到达一个下一跳后增加 1；
|
||||||
- `retry_count`：当前消息已发生的重试次数，用于控制重试上限并统计额外流量；
|
- `retry_count`：当前消息已发生的重试次数，用于控制重试上限并统计额外流量；
|
||||||
- 时间字段（如 `timestamp`、创建时间、到达时间和超时时间）：用于计算排队时间、处理时延、链路时延和超时；
|
- 时间字段（如 `timestamp`、创建时间、到达时间和超时时间）：用于计算排队时间、处理时延、链路时延和超时；
|
||||||
- `payload`：消息承载的业务内容。在网络性能模型中通常不展开其具体内容，只通过 `message_type` 和 `message_size` 描述其处理类别和传输开销。
|
- `payload`：消息承载的业务内容。在网络性能模型中通常不展开其具体内容，只通过 `message_type` 和 `message_size` 描述其处理类别和传输开销。
|
||||||
|
|
||||||
其中，`source`、`destination`、`message_type`、`message_size` 和 `priority` 是消息的基本属性；`path`、$h$、队列位置和时间字段是消息传播过程中的动态状态。若只研究消息传播和流量，可以省略 `payload`；若需要恢复任务分解和重试关系，则应保留 `task_id`、`message_id`、`parent_message_id` 和 `retry_count`。
|
其中，`source`、`destination`、`message_type`、`message_size` 和 `priority` 是消息的基本属性；`next_hop`、`path`、$h$、队列位置和时间字段是消息传播过程中的动态状态。`message_id` 标识逻辑消息，`attempt_id` 标识一次实际发送尝试。若只研究消息传播、任务分解、请求响应和重试关系，则应保留 `task_id`、`message_id`、`parent_message_id`、`correlation_id`、`attempt_id` 和 `retry_count`。
|
||||||
|
|
||||||
例如，路径为 $A\rightarrow B\rightarrow C\rightarrow D$ 时，消息位于 $C$，则 $h=2$。
|
例如，路径为 $A\rightarrow B\rightarrow C\rightarrow D$ 时，消息位于 $C$，则 $h=2$。
|
||||||
|
|
||||||
消息类型可以包括：新任务、子任务请求、工具调用、工具返回结果、Agent 间协商、最终结果、错误通知和重试请求。
|
消息类型可以包括：新任务、子任务请求、工具调用、工具返回结果、Agent 间协商、最终结果、错误通知和重试请求。
|
||||||
|
|
||||||
### 2.3 节点状态
|
### 2.3 节点状态与消息处理状态
|
||||||
|
|
||||||
节点状态描述节点当前正在做什么：
|
需要区分两类状态：
|
||||||
|
|
||||||
|
- 节点资源状态 $q_v$：描述节点整体是否有可用处理资源。它可以表示为 `{Idle, Busy, Saturated, Failed}`，其中 `Busy` 表示至少有一个活跃处理上下文，`Saturated` 表示活跃处理数达到 `max_concurrency`；
|
||||||
|
- 消息处理状态 $q_m$：描述某条消息或某个任务处理上下文当前正在做什么，例如接收、排队、思考、调用工具、等待结果、发送结果、失败或重试。
|
||||||
|
|
||||||
|
因此，下面的通用状态机是“每条消息/任务处理上下文”的状态机，而不是整个节点在任意时刻只能拥有一个状态。一个节点可以在同一时刻拥有多个 $q_m$，但其活跃处理上下文数量不能超过 `max_concurrency`。
|
||||||
|
|
||||||
$$
|
$$
|
||||||
状态=\{空闲, 接收消息, 思考, 调用工具, 等待结果, 发送结果, 失败, 重试\}
|
q_m=\{空闲, 接收消息, 排队, 思考, 调用工具, 等待结果, 发送结果, 失败, 重试\}
|
||||||
$$
|
$$
|
||||||
|
|
||||||
典型状态变化如下：
|
典型状态变化如下：
|
||||||
@@ -103,7 +111,7 @@ stateDiagram-v2
|
|||||||
|
|
||||||
### 3.1 外层自动机：消息在网络中的传播位置
|
### 3.1 外层自动机：消息在网络中的传播位置
|
||||||
|
|
||||||
外层状态不再表示固定的节点类型，而表示消息当前位于哪个节点、正在执行什么交互，以及下一步根据局部拓扑选择什么目标节点。
|
外层自动机只负责消息在网络中的传播：确定逻辑目的地、选择当前下一跳、安排链路传输和处理到达事件。它不负责决定目标节点收到消息后执行 `Think`、`CallTool` 或 `Split` 等业务状态。
|
||||||
|
|
||||||
可以将外层状态理解为：
|
可以将外层状态理解为：
|
||||||
|
|
||||||
@@ -111,6 +119,8 @@ $$
|
|||||||
大状态=(当前节点, 当前交互上下文, 当前路径位置)
|
大状态=(当前节点, 当前交互上下文, 当前路径位置)
|
||||||
$$
|
$$
|
||||||
|
|
||||||
|
外层每次只选择一个 `next_hop`。候选下一跳必须属于当前节点的可用出邻居集合；最终 `destination` 可以是多跳之外的节点。若采用预先计算的路由，`path` 保存完整计划路径；若采用逐跳路由，`path` 保存已经确认的实际路径，并在每次发送后追加新的下一跳。消息到达 `destination` 后，不再执行普通转发；响应消息应作为新的逻辑消息沿反向或重新计算的路径传播。
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
stateDiagram-v2
|
stateDiagram-v2
|
||||||
[*] --> Node_A: 外部任务到达
|
[*] --> Node_A: 外部任务到达
|
||||||
@@ -135,7 +145,7 @@ Node_A -> Node_B -> Node_C -> Node_B -> Node_A
|
|||||||
|
|
||||||
### 3.2 内层自动机：所有节点共享的通用行为
|
### 3.2 内层自动机：所有节点共享的通用行为
|
||||||
|
|
||||||
内层自动机不按节点类型拆成多套固定流程，而是使用一套通用状态机。节点是否能够执行某个分支，由它的能力、周围拓扑、任务上下文和当前负载决定。
|
内层自动机绑定到一个具体的消息或任务处理上下文，不按节点类型拆成多套固定流程，而是使用一套通用状态机。节点是否能够执行某个分支，由节点能力、周围拓扑、任务上下文、当前负载和可用并发槽决定。
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
stateDiagram-v2
|
stateDiagram-v2
|
||||||
@@ -175,24 +185,35 @@ stateDiagram-v2
|
|||||||
- 邻居负载较高时，`CallAgent` 可能选择其他邻居或进入等待；
|
- 邻居负载较高时，`CallAgent` 可能选择其他邻居或进入等待；
|
||||||
- 同一个节点在不同任务阶段，可以表现出协调、执行、查询或转发等不同功能。
|
- 同一个节点在不同任务阶段，可以表现出协调、执行、查询或转发等不同功能。
|
||||||
|
|
||||||
### 3.3 消息的完整状态
|
### 3.3 消息处理上下文的完整状态
|
||||||
|
|
||||||
一条消息关联的最小节点状态可以表示为：
|
一条消息或任务处理上下文的最小状态可以表示为：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
(当前节点, 当前节点内部状态)
|
(消息或任务上下文, 当前节点, 当前消息处理状态)
|
||||||
```
|
```
|
||||||
|
|
||||||
例如：
|
例如：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
(Node_A, Think)
|
(M_1, Node_A, Think)
|
||||||
(Node_A, CallTool)
|
(M_1, Node_A, CallTool)
|
||||||
(Node_B, Query)
|
(M_2, Node_B, Query)
|
||||||
(Node_A, Wait)
|
(M_1, Node_A, Wait)
|
||||||
(Node_A, Send)
|
(M_1, Node_A, Send)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
同一个节点可以同时存在多个处理上下文，例如：
|
||||||
|
|
||||||
|
```text
|
||||||
|
Node_A:
|
||||||
|
(M_1, Think)
|
||||||
|
(M_2, Wait)
|
||||||
|
(M_3, Queue)
|
||||||
|
```
|
||||||
|
|
||||||
|
其中活跃上下文数量受 `max_concurrency` 限制，`M_3` 仍在等待资源。节点级状态由资源占用汇总得到：没有活跃上下文时为 `Idle`，有活跃上下文但未达到上限时为 `Busy`，达到 `max_concurrency` 时为 `Saturated`。具体的队列操作由第 7 章定义。
|
||||||
|
|
||||||
### 3.4 带流量信息的状态机
|
### 3.4 带流量信息的状态机
|
||||||
|
|
||||||
第 3.2 节给出了通用的内层自动机。本节在其状态转移上增加消息数量、消息大小、入站/出站字节数、超时概率和重试流量。下面的图是“工具调用或直接返回”场景的简化子集，不是独立于内层自动机的新模型：
|
第 3.2 节给出了通用的内层自动机。本节在其状态转移上增加消息数量、消息大小、入站/出站字节数、超时概率和重试流量。下面的图是“工具调用或直接返回”场景的简化子集，不是独立于内层自动机的新模型：
|
||||||
@@ -214,6 +235,8 @@ stateDiagram-v2
|
|||||||
|
|
||||||
`Split`、`CallAgent`、`Query`、`Forward` 和 `Failed` 等状态并未消失。在具体场景中，应以同样的方式为这些状态转移增加消息数量、大小、目标节点、失败概率和流量变化。例如，`Think -> Split` 产生多个子任务消息，`Think -> Query` 产生查询请求，`Think -> Forward` 产生转发流量。`Done` 仅作为流程完成标记，不作为独立的节点内部状态，正式状态转移使用 `Send -> Idle`。
|
`Split`、`CallAgent`、`Query`、`Forward` 和 `Failed` 等状态并未消失。在具体场景中，应以同样的方式为这些状态转移增加消息数量、大小、目标节点、失败概率和流量变化。例如，`Think -> Split` 产生多个子任务消息，`Think -> Query` 产生查询请求，`Think -> Forward` 产生转发流量。`Done` 仅作为流程完成标记，不作为独立的节点内部状态，正式状态转移使用 `Send -> Idle`。
|
||||||
|
|
||||||
|
在一个节点上，`Idle`、`Receive`、`Think` 等图中的状态表示某个消息处理上下文的状态。节点整体状态由并发资源汇总得到，不应把多个上下文强行合并成一个唯一的业务状态。
|
||||||
|
|
||||||
其中：
|
其中：
|
||||||
|
|
||||||
- $p_{tool}$：思考后调用工具的概率；
|
- $p_{tool}$：思考后调用工具的概率；
|
||||||
@@ -282,13 +305,13 @@ flowchart TD
|
|||||||
对于一条具体消息，完整状态可以表示为：
|
对于一条具体消息，完整状态可以表示为：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
(当前节点, 当前节点内部状态, 任务上下文, 消息路径, 消息属性, 队列负载)
|
(当前节点, 节点资源状态, 当前消息处理状态, 任务上下文, 消息路径, 消息属性, 队列负载)
|
||||||
```
|
```
|
||||||
|
|
||||||
仿真循环可以概括为：
|
仿真循环可以概括为：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
外层自动机选择下一节点
|
外层自动机选择下一跳
|
||||||
-> 目标节点进入内层自动机
|
-> 目标节点进入内层自动机
|
||||||
-> 内层状态转移产生处理动作和新消息
|
-> 内层状态转移产生处理动作和新消息
|
||||||
-> 更新队列、链路路径和流量
|
-> 更新队列、链路路径和流量
|
||||||
@@ -409,7 +432,7 @@ $$
|
|||||||
|
|
||||||
### 4.7 处理速度 $\mu$
|
### 4.7 处理速度 $\mu$
|
||||||
|
|
||||||
$\mu$ 表示节点单位时间能够处理的消息数量。
|
$\mu$ 表示节点总的单位时间处理能力。存在并发槽时，不能默认将单槽处理速度直接乘以 `max_concurrency`；如果日志记录的是单个并发槽速度，则还需要额外建模并发竞争、资源共享和不同消息类型的处理时间。
|
||||||
|
|
||||||
如果：
|
如果：
|
||||||
|
|
||||||
@@ -425,7 +448,9 @@ $$
|
|||||||
|
|
||||||
消息会不断进入队列，最终导致延迟和拥塞。
|
消息会不断进入队列，最终导致延迟和拥塞。
|
||||||
|
|
||||||
## 5. 从小规模日志估计自动机参数
|
## 5. 参数估计、数据采集与数据量要求
|
||||||
|
|
||||||
|
本章回答“模型参数从哪里来、需要记录什么以及需要多少数据”。第 6 章再定义这些条件变量和参数如何进入状态转移模型；因此，本章讨论的是参数的可观测性、估计方法和数据充分性，而不是新增状态机结构。
|
||||||
|
|
||||||
状态机中的概率和持续时间，主要通过小规模实验或真实运行日志估计。这里的“训练”首先是统计参数，不一定需要复杂的机器学习模型。
|
状态机中的概率和持续时间，主要通过小规模实验或真实运行日志估计。这里的“训练”首先是统计参数，不一定需要复杂的机器学习模型。
|
||||||
|
|
||||||
@@ -441,9 +466,12 @@ $$
|
|||||||
| `task_id` | 还原一次完整任务的执行过程 |
|
| `task_id` | 还原一次完整任务的执行过程 |
|
||||||
| `message_id` | 追踪一条具体消息 |
|
| `message_id` | 追踪一条具体消息 |
|
||||||
| `parent_message_id` | 判断当前消息由哪条消息产生 |
|
| `parent_message_id` | 判断当前消息由哪条消息产生 |
|
||||||
|
| `correlation_id` | 匹配请求、响应、错误和超时事件 |
|
||||||
|
| `attempt_id` | 区分同一逻辑请求的不同发送尝试 |
|
||||||
| `message_type` | 区分任务、工具调用、工具返回、错误和重试 |
|
| `message_type` | 区分任务、工具调用、工具返回、错误和重试 |
|
||||||
| `source` | 记录发送节点 |
|
| `source` | 记录发送节点 |
|
||||||
| `destination` | 记录接收节点 |
|
| `destination` | 记录接收节点 |
|
||||||
|
| `next_hop` | 记录本次传输实际到达的下一跳节点 |
|
||||||
| `node_capabilities` | 记录节点当前具备的能力 |
|
| `node_capabilities` | 记录节点当前具备的能力 |
|
||||||
| `local_topology` | 记录节点的邻居、连接数量、距离和邻居负载 |
|
| `local_topology` | 记录节点的邻居、连接数量、距离和邻居负载 |
|
||||||
| `state_before` | 记录状态转移前的状态 |
|
| `state_before` | 记录状态转移前的状态 |
|
||||||
@@ -511,6 +539,8 @@ $$
|
|||||||
|
|
||||||
$D_{think}$ 表示从日志中得到的思考时间分布。仿真时从该分布中抽取一个具体持续时间，而不是每次固定使用同一个数值。
|
$D_{think}$ 表示从日志中得到的思考时间分布。仿真时从该分布中抽取一个具体持续时间，而不是每次固定使用同一个数值。
|
||||||
|
|
||||||
|
端到端延迟拆分为排队时间、节点处理时间、链路传输时间和等待外部结果时间。`Think` 的持续时间只表示节点实际思考/处理时间；`Wait` 的持续时间表示从请求发送到匹配响应或超时的等待时间；队列等待时间和链路传输时间单独记录，不能重复计入状态持续时间。
|
||||||
|
|
||||||
#### 消息大小分布
|
#### 消息大小分布
|
||||||
|
|
||||||
工具请求、工具结果和最终结果应分别统计：
|
工具请求、工具结果和最终结果应分别统计：
|
||||||
@@ -537,12 +567,16 @@ $$
|
|||||||
P(destination\mid 当前节点,消息类型,网络状态)
|
P(destination\mid 当前节点,消息类型,网络状态)
|
||||||
$$
|
$$
|
||||||
|
|
||||||
|
目标选择先构造满足能力、可达性和资源约束的候选集合，再在候选集合内抽样；不可达或能力不满足的节点概率为 0。若消息需要多跳传播，目标选择确定 `destination`，路由模型再为每一步确定 `next_hop`。没有可行目标时，消息按照配置进入等待、改路由、重试或 `Failed`。
|
||||||
|
|
||||||
#### 超时和重试概率
|
#### 超时和重试概率
|
||||||
|
|
||||||
$$
|
$$
|
||||||
P_{timeout}=0.05
|
P_{timeout}=0.05
|
||||||
$$
|
$$
|
||||||
|
|
||||||
|
`p_timeout` 表示一次发送尝试在规定超时时间内未收到匹配响应的概率；超时时间由请求的 `timeout_at` 或条件超时分布确定，不与处理失败概率混为一谈。每次重试都创建新的 `attempt_id`，递增同一逻辑请求的 `retry_count`，保留原有 `correlation_id`，并重新执行目标选择和链路传输。达到 `max_retries` 后，当前请求进入 `Failed`，并按配置向父上下文发送错误消息或结束任务。
|
||||||
|
|
||||||
### 5.5 条件参数
|
### 5.5 条件参数
|
||||||
|
|
||||||
不能简单地让所有节点共用一套概率。状态转移通常与节点能力、局部拓扑、任务类型、消息类型和系统负载有关：
|
不能简单地让所有节点共用一套概率。状态转移通常与节点能力、局部拓扑、任务类型、消息类型和系统负载有关：
|
||||||
@@ -560,7 +594,7 @@ $$
|
|||||||
高负载时超时概率：0.15
|
高负载时超时概率：0.15
|
||||||
```
|
```
|
||||||
|
|
||||||
### 5.6 十个节点数据的适用边界
|
### 5.6 小规模日志可以估计的参数
|
||||||
|
|
||||||
十个节点的数据可以用于学习：
|
十个节点的数据可以用于学习：
|
||||||
|
|
||||||
@@ -580,11 +614,80 @@ $$
|
|||||||
- 高负载下的处理速度下降；
|
- 高负载下的处理速度下降；
|
||||||
- 拥塞引发的超时和重试放大。
|
- 拥塞引发的超时和重试放大。
|
||||||
|
|
||||||
因此，小规模日志应先用于建立基础行为模型，再通过压测或更大规模实验校准容量、队列、延迟和重试模型。
|
因此，小规模日志适合先建立基础行为模型；容量、队列、延迟和重试模型所需的数据规模与适用边界见第 5.8～5.10 节。
|
||||||
|
|
||||||
## 6. 分层模型、开放类别与泛化
|
### 5.7 数据量估算方法
|
||||||
|
|
||||||
任务类型、任务阶段和节点能力不应被理解为只能从训练数据中固定选择的封闭标签。需要区分两件事：
|
按照本文模型的定义，所谓“训练”主要是从运行日志中估计状态转移概率、状态持续时间、消息大小、目标节点选择、失败概率和重试行为等参数，并不一定需要训练大规模深度学习模型。因此，数据量主要取决于需要区分的条件组合数量，而不是网络节点总数。
|
||||||
|
|
||||||
|
消息级日志、状态事件和任务级轨迹应分别统计：
|
||||||
|
|
||||||
|
- **消息级记录**：描述一条消息的产生、传输、排队、处理、完成、失败或重试；
|
||||||
|
- **状态事件记录**：描述节点一次状态转移及其持续时间，例如 `Think -> CallTool`；
|
||||||
|
- **任务级轨迹**：通过 `task_id` 和 `parent_message_id` 连接同一任务产生的多条消息，用于恢复任务拆分、协作和返回过程。
|
||||||
|
|
||||||
|
对于当前模型，状态事件数通常比完整任务数更适合作为参数估计的数据量指标。
|
||||||
|
|
||||||
|
设需要区分的条件组合数量为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
K=K_{task}\times K_{phase}\times K_{message}\times K_{capability}\times K_{load}\times K_{topology}
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中各项分别表示任务类型、任务阶段、消息类型、节点能力类别、负载区间和局部拓扑类别的数量。若每个条件组合至少需要 $n$ 次有效状态事件，则：
|
||||||
|
|
||||||
|
$$
|
||||||
|
N_{event}\approx K\times n
|
||||||
|
$$
|
||||||
|
|
||||||
|
实际采集量还应考虑数据清洗、无效记录和验证集划分，建议将结果扩大约 $1.5\sim2$ 倍。分层模型通过共享全局参数和上级层次参数，可以降低每个条件组合所需的独立样本量。
|
||||||
|
|
||||||
|
### 5.8 推荐数据量级
|
||||||
|
|
||||||
|
| 目标 | 完整任务数 | 消息或状态事件数 | 适用范围 |
|
||||||
|
| --- | ---: | ---: | --- |
|
||||||
|
| 原型验证 | $10^3\sim10^4$ | $10^4\sim10^5$ | 验证状态机、消息路径和仿真流程 |
|
||||||
|
| 基础可用 | $10^4\sim10^5$ | $10^5\sim10^6$ | 估计主要状态转移、处理时间和消息大小 |
|
||||||
|
| 较稳定预测 | $10^5\sim10^6$ | $10^6\sim10^7$ | 覆盖不同任务、负载、拓扑和失败场景 |
|
||||||
|
| 大规模容量评估 | $10^6$ 以上 | $10^7$ 以上 | 预测热点、拥塞、超时和重试放大 |
|
||||||
|
|
||||||
|
对于目前的十节点模型，建议至少收集 $10^5\sim10^6$ 条消息或状态事件，或者 $10^4\sim10^5$ 个完整任务轨迹。若重点研究拥塞、失败和重试，则建议达到 $10^6\sim10^7$ 条事件，并确保覆盖高负载和异常场景。
|
||||||
|
|
||||||
|
普通状态转移概率每个重要条件组合至少需要 100～300 次状态结束事件；状态持续时间和消息大小分布建议每个主要类别有 300～1000 个样本。失败、超时和重试等稀有事件应至少观察到 50～100 次；若超时率约为 5%，通常需要约 2000 次相关请求才能得到相对稳定的估计。
|
||||||
|
|
||||||
|
### 5.9 十节点场景示例与运行速率换算
|
||||||
|
|
||||||
|
假设需要区分 5 种任务类型、5 个任务阶段、5 类消息、2 类节点能力和 3 个负载区间，暂不单独划分拓扑类别，则：
|
||||||
|
|
||||||
|
$$
|
||||||
|
K=5\times5\times5\times2\times3=750
|
||||||
|
$$
|
||||||
|
|
||||||
|
若每个条件组合收集 200 次状态事件，基础需求为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
750\times200=150000
|
||||||
|
$$
|
||||||
|
|
||||||
|
考虑数据清洗、验证集和异常场景后，建议收集约 $2\times10^5\sim5\times10^5$ 条有效事件。
|
||||||
|
|
||||||
|
若系统总消息到达率为 $\lambda$，连续观测时间为 $T$，则：
|
||||||
|
|
||||||
|
$$
|
||||||
|
N_{message}\approx\lambda T
|
||||||
|
$$
|
||||||
|
|
||||||
|
例如 $\lambda=100$ 条/秒时，1 小时约产生 36 万条消息，1 天约产生 864 万条消息。这里的 $\lambda$ 必须明确是整个网络的总到达率，还是每个节点的到达率。消息数量不等于完整任务数量，因为一个任务可能通过拆分、工具调用和重试产生多条消息。
|
||||||
|
|
||||||
|
### 5.10 参数回退、校准与数据适用边界
|
||||||
|
|
||||||
|
如果某个条件组合的样本过少，应使用分层模型向任务类型、任务阶段、节点能力或全局参数回退，并记录 `parameter_source` 和 `confidence`。新增任务或节点类型可以先继承相似类别的参数，再用运行数据进行增量校准。
|
||||||
|
|
||||||
|
十个节点的日志可以用于学习节点内部状态转换、工具调用概率、处理时间、消息大小分布和基本调用链结构，也可以初步估计失败与重试行为。但它不能单独保证模型能够准确预测大规模网络中的热点竞争、链路拥塞和容量变化，因此还需要压测或更大规模实验进行校准。
|
||||||
|
|
||||||
|
## 6. 条件状态转移与分层参数模型
|
||||||
|
|
||||||
|
本章只定义模型如何使用任务类型、任务阶段、节点能力、局部拓扑和系统负载等条件变量，不讨论日志应该采集多少或参数如何校准；这些内容统一放在第 5 章。需要区分两件事：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
类别定义：系统如何描述任务、阶段和节点能力
|
类别定义：系统如何描述任务、阶段和节点能力
|
||||||
@@ -593,7 +696,7 @@ $$
|
|||||||
|
|
||||||
类别可以在运行时新增，但新增类别是否能够立即得到准确行为，取决于它是否能够用已有特征表示，以及是否有足够的新数据进行校准。
|
类别可以在运行时新增，但新增类别是否能够立即得到准确行为，取决于它是否能够用已有特征表示，以及是否有足够的新数据进行校准。
|
||||||
|
|
||||||
#### 分层参数
|
### 6.1 分层参数结构
|
||||||
|
|
||||||
不为每一种任务类型、任务阶段和节点分别建立完全独立的状态机，而是让不同层次共享一部分基础参数：
|
不为每一种任务类型、任务阶段和节点分别建立完全独立的状态机，而是让不同层次共享一部分基础参数：
|
||||||
|
|
||||||
@@ -617,7 +720,7 @@ $$
|
|||||||
|
|
||||||
其中：
|
其中：
|
||||||
|
|
||||||
- $q$：节点当前内部状态；
|
- $q$：当前消息处理上下文的状态，即本文的 $q_m$；
|
||||||
- $c$：任务类型、任务阶段和其他业务上下文；
|
- $c$：任务类型、任务阶段和其他业务上下文；
|
||||||
- $x_v$：节点能力和资源；
|
- $x_v$：节点能力和资源；
|
||||||
- $z_v$：局部拓扑特征；
|
- $z_v$：局部拓扑特征；
|
||||||
@@ -645,7 +748,7 @@ $$
|
|||||||
|
|
||||||
这样，同一个 `Think` 状态在不同任务阶段可以具有不同的处理时间、调用工具概率、输出大小和下游消息数量。
|
这样，同一个 `Think` 状态在不同任务阶段可以具有不同的处理时间、调用工具概率、输出大小和下游消息数量。
|
||||||
|
|
||||||
#### 分层模型的一次计算过程
|
### 6.2 条件状态转移的计算过程
|
||||||
|
|
||||||
在仿真中，状态转移不是直接查找一个固定概率，而是对每一条候选转移分别计算分数，再转换为概率。一次计算可以分为以下步骤。
|
在仿真中，状态转移不是直接查找一个固定概率，而是对每一条候选转移分别计算分数，再转换为概率。一次计算可以分为以下步骤。
|
||||||
|
|
||||||
@@ -655,7 +758,7 @@ $$
|
|||||||
|
|
||||||
```text
|
```text
|
||||||
节点：Node_B
|
节点：Node_B
|
||||||
当前内部状态：Think
|
当前消息处理状态：Think
|
||||||
任务类型：recruitment
|
任务类型：recruitment
|
||||||
任务阶段：detailed_evaluation
|
任务阶段：detailed_evaluation
|
||||||
消息类型：resume_request
|
消息类型：resume_request
|
||||||
@@ -816,7 +919,7 @@ Wait → Retry → CallTool → Wait
|
|||||||
|
|
||||||
每次重试都要重新计算请求消息、目标节点和链路流量，因此会产生额外流量。
|
每次重试都要重新计算请求消息、目标节点和链路流量，因此会产生额外流量。
|
||||||
|
|
||||||
#### 分层计算的伪代码
|
### 6.3 分层计算的伪代码
|
||||||
|
|
||||||
```text
|
```text
|
||||||
处理事件(event):
|
处理事件(event):
|
||||||
@@ -840,6 +943,8 @@ Wait → Retry → CallTool → Wait
|
|||||||
安排完成、返回、超时或重试事件
|
安排完成、返回、超时或重试事件
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 6.4 新类别的模型表达
|
||||||
|
|
||||||
#### 新增任务类型
|
#### 新增任务类型
|
||||||
|
|
||||||
新任务类型不应只依赖一个从未见过的离散名称，而应尽量使用可解释特征描述：
|
新任务类型不应只依赖一个从未见过的离散名称，而应尽量使用可解释特征描述：
|
||||||
@@ -941,6 +1046,8 @@ confidence = low
|
|||||||
|
|
||||||
> 消息已经到达某个节点或链路，但当前没有足够的处理能力，因此暂时等待的消息集合。
|
> 消息已经到达某个节点或链路，但当前没有足够的处理能力，因此暂时等待的消息集合。
|
||||||
|
|
||||||
|
本模型将 `Q_v(t)` 定义为“已到达节点、但尚未分配到活跃处理资源的消息数”。正在 `Think`、`CallTool` 或 `Wait` 中的处理上下文不计入 `Q_v(t)`，但会计入节点的活跃并发数和资源占用 `r_v(t)`。链路传输中的消息不计入节点队列；若要建模链路缓冲区，则另设 `Q_e(t)`。
|
||||||
|
|
||||||
现实系统中，等待消息可能存在于独立消息中间件、服务内部、Agent mailbox、模型服务、网络缓冲区或外部服务请求队列中。因此，即使 Agent 之间采用直接同步调用，系统中仍可能存在隐性的等待位置：
|
现实系统中，等待消息可能存在于独立消息中间件、服务内部、Agent mailbox、模型服务、网络缓冲区或外部服务请求队列中。因此，即使 Agent 之间采用直接同步调用，系统中仍可能存在隐性的等待位置：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
@@ -962,15 +1069,19 @@ Agent A -> 网络缓冲区 -> Agent B服务队列 -> 线程池 -> Agent B执行
|
|||||||
|
|
||||||
队列增长会带来消息等待时间增加、节点处理延迟增加、请求超时、重试流量增加、限流或请求拒绝，以及热点节点和链路拥塞。
|
队列增长会带来消息等待时间增加、节点处理延迟增加、请求超时、重试流量增加、限流或请求拒绝，以及热点节点和链路拥塞。
|
||||||
|
|
||||||
|
队列采用有界、按 `priority` 优先且同优先级按到达顺序处理的调度规则；具体优先级范围由系统配置决定。节点有 $C_v=\text{max\_concurrency}$ 个并发槽。只有当活跃上下文数小于 $C_v$ 时，队首消息才能从 `Queue` 进入 `Receive` 或 `Think`，并使活跃上下文数加一。默认情况下，进入 `Wait` 的上下文仍占用一个并发槽，因为它仍需要维护请求、超时和响应匹配；若实际系统允许等待期间释放资源，应配置 `wait_holds_slot=false`，并在恢复时重新申请并发槽。上下文进入 `Send` 完成、`Failed` 或其他终止状态后释放一个并发槽，并立即尝试调度队列中的下一条消息。
|
||||||
|
|
||||||
|
本文采用离散事件仿真作为正式实现方式。固定窗口公式只用于统计和校验，不作为状态推进的唯一机制。
|
||||||
|
|
||||||
### 7.3 队列动态
|
### 7.3 队列动态
|
||||||
|
|
||||||
节点或链路通常不能立即处理所有消息，因此需要维护等待队列或缓冲区：
|
节点或链路通常不能立即处理所有消息，因此需要维护等待队列或缓冲区。对节点队列，在固定统计窗口中可以使用：
|
||||||
|
|
||||||
$$
|
$$
|
||||||
Q_v(t+\Delta t)=\max\left(0, Q_v(t)+A_v(t,\Delta t)-D_v(t,\Delta t)\right)
|
Q_v(t+\Delta t)=\max\left(0, Q_v(t)+A_v(t,\Delta t)-D_v(t,\Delta t)\right)
|
||||||
$$
|
$$
|
||||||
|
|
||||||
其中 $Q_v(t)$ 是当前队列消息数，$A_v$ 是时间窗口内到达的消息数，$D_v$ 是时间窗口内处理完成的消息数，并且应满足 $D_v(t,\Delta t)\leq Q_v(t)+A_v(t,\Delta t)$。队列还可以设置容量上限：
|
其中 $Q_v(t)$ 是等待队列消息数，$A_v$ 是窗口内到达并需要排队的消息数，$D_v$ 是窗口内从等待队列取出并开始处理的消息数。处理完成数不直接等同于队列减少数；消息开始处理时才离开等待队列。应满足 $D_v(t,\Delta t)\leq Q_v(t)+A_v(t,\Delta t)$。在离散事件仿真中，每次消息到达使 `Q_v` 加一（若无可用资源），每次分配处理资源使 `Q_v` 减一。
|
||||||
|
|
||||||
$$
|
$$
|
||||||
Q_v(t)\leq Q_v^{max}
|
Q_v(t)\leq Q_v^{max}
|
||||||
@@ -998,6 +1109,8 @@ $$
|
|||||||
B_v=B_v^{in}+B_v^{out}
|
B_v=B_v^{in}+B_v^{out}
|
||||||
$$
|
$$
|
||||||
|
|
||||||
|
链路按有向边分别统计：$B_{u\rightarrow v}(T)$ 只包含从 $u$ 到 $v$ 的消息字节数；如果需要双向链路总量，则另定义 $B_{\{u,v\}}(T)=B_{u\rightarrow v}(T)+B_{v\rightarrow u}(T)$。节点的入站/出站流量默认包含外部系统通信；若只研究节点间通信，则必须将外部系统单独作为节点或单独统计，不能在同一结果中混用两种口径。
|
||||||
|
|
||||||
## 9. 状态机如何指导最终模拟
|
## 9. 状态机如何指导最终模拟
|
||||||
|
|
||||||
状态机不是直接给出最终流量，而是作为模拟器的行为规则。模拟器不断执行“读取事件、状态转移、生成消息、更新队列和统计流量”的循环。
|
状态机不是直接给出最终流量，而是作为模拟器的行为规则。模拟器不断执行“读取事件、状态转移、生成消息、更新队列和统计流量”的循环。
|
||||||
@@ -1024,18 +1137,18 @@ Node_C：具备工具执行能力，状态 = Idle
|
|||||||
|
|
||||||
### 9.2 事件处理循环
|
### 9.2 事件处理循环
|
||||||
|
|
||||||
模拟器按照时间顺序处理事件：
|
模拟器按照时间顺序处理事件；同一时间戳下按“消息到达 → 状态完成/资源释放 → 响应匹配 → 超时检查 → 重试或失败”的顺序处理。已经匹配到响应的请求会取消其未触发的超时事件。
|
||||||
|
|
||||||
```text
|
```text
|
||||||
1. 取出时间最早的事件
|
1. 取出时间最早的事件
|
||||||
2. 找到相关节点
|
2. 找到相关节点、消息处理上下文和 `correlation_id`
|
||||||
3. 更新节点队列和资源状态
|
3. 更新消息到达、节点队列和并发资源状态
|
||||||
4. 根据当前状态和参数选择下一状态
|
4. 根据当前消息处理状态和参数选择下一状态
|
||||||
5. 从持续时间分布中抽取处理时间
|
5. 从条件持续时间分布中抽取处理时间
|
||||||
6. 根据状态转移产生新的消息
|
6. 根据状态转移产生新的逻辑消息或发送尝试
|
||||||
7. 按消息路径更新节点和链路流量
|
7. 按实际 `next_hop` 更新节点和有向链路流量
|
||||||
8. 将新消息放入目标节点队列
|
8. 将到达消息放入目标节点队列，或在有资源时直接分配并发槽
|
||||||
9. 安排后续的完成、超时或重试事件
|
9. 安排后续的状态完成、响应匹配、超时、重试或失败事件
|
||||||
```
|
```
|
||||||
|
|
||||||
### 9.3 一个任务的模拟过程
|
### 9.3 一个任务的模拟过程
|
||||||
@@ -1048,6 +1161,8 @@ Idle -> Receive -> Think -> Split
|
|||||||
|
|
||||||
`Split` 状态根据参数 $F=3$ 产生三条子任务消息。每条消息根据局部拓扑和目标选择规则进入一个邻居节点的队列，同时记录出站消息数和字节流量。
|
`Split` 状态根据参数 $F=3$ 产生三条子任务消息。每条消息根据局部拓扑和目标选择规则进入一个邻居节点的队列，同时记录出站消息数和字节流量。
|
||||||
|
|
||||||
|
拆分后的父上下文进入 `Wait` 后，必须记录待完成子任务集合。只有在所有必需子任务返回、某个可配置的部分完成条件满足，或任务因失败/超时达到终止条件时，父上下文才从 `Wait` 返回 `Think` 或进入 `Failed`。单个子任务返回不会自动使父上下文继续执行。
|
||||||
|
|
||||||
目标节点随后进入：
|
目标节点随后进入：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
@@ -1157,6 +1272,8 @@ Node_C：Idle，队列 = 0
|
|||||||
|
|
||||||
### 10.3 三节点状态变化和流量时序图
|
### 10.3 三节点状态变化和流量时序图
|
||||||
|
|
||||||
|
本节图中的“Node_X 状态”表示该节点当前示例任务对应的消息处理上下文状态；如果节点同时处理其他任务，还应分别记录其他上下文，不能据此推断节点只有一个全局业务状态。图中的双向连线仅用于展示请求和响应方向，正式流量按第 8 节的有向链路口径统计。
|
||||||
|
|
||||||
下面用连续的网络快照展示一次任务的演化过程。每一个步骤都重新绘制相同的三节点拓扑，并在当前图中标注：
|
下面用连续的网络快照展示一次任务的演化过程。每一个步骤都重新绘制相同的三节点拓扑，并在当前图中标注：
|
||||||
|
|
||||||
- 节点当前状态和队列长度；
|
- 节点当前状态和队列长度；
|
||||||
@@ -1391,6 +1508,8 @@ Node_A: Idle → Receive → Think → Send → Idle
|
|||||||
|
|
||||||
本次任务完成。
|
本次任务完成。
|
||||||
|
|
||||||
|
如果 `Node_A` 代表面向外部系统的入口节点，则任务完成后还应生成一条 `Node_A → 外部系统` 的最终结果消息。该消息大小必须单独配置；本节原有的数值汇总只统计外部输入和节点间通信，未把这条未给出大小的外部结果消息计入。若统计节点总出站流量，则应在 `Node_A` 出站流量中另加该消息，并单独标注外部通信。
|
||||||
|
|
||||||
### 10.8 一次任务的汇总结果
|
### 10.8 一次任务的汇总结果
|
||||||
|
|
||||||
消息路径为：
|
消息路径为：
|
||||||
@@ -1402,8 +1521,17 @@ Node_A: Idle → Receive → Think → Send → Idle
|
|||||||
各链路累计流量为：
|
各链路累计流量为：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
A-B：3 KB + 2 KB = 5 KB
|
A→B：3 KB
|
||||||
B-C：1 KB + 4 KB = 5 KB
|
B→A：2 KB
|
||||||
|
B→C：1 KB
|
||||||
|
C→B：4 KB
|
||||||
|
```
|
||||||
|
|
||||||
|
若汇总为无向链路总量，则：
|
||||||
|
|
||||||
|
```text
|
||||||
|
{A,B}：3 KB + 2 KB = 5 KB
|
||||||
|
{B,C}：1 KB + 4 KB = 5 KB
|
||||||
```
|
```
|
||||||
|
|
||||||
节点处理得到的通信流量为：
|
节点处理得到的通信流量为：
|
||||||
@@ -1429,15 +1557,17 @@ Node_B: Wait → Retry → CallTool → Wait
|
|||||||
如果外部任务到达率为 $\lambda_{ext}$，且每个任务都经过上述路径，则在不考虑失败和重试时：
|
如果外部任务到达率为 $\lambda_{ext}$，且每个任务都经过上述路径，则在不考虑失败和重试时：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
A-B 平均消息流量 ≈ λ × (3 + 2) KB
|
A→B 平均消息流量 ≈ λ × 3 KB
|
||||||
B-C 平均消息流量 ≈ λ × (0.7 × (1 + 4)) KB
|
B→A 平均消息流量 ≈ λ × 2 KB
|
||||||
|
B→C 平均消息流量 ≈ λ × (0.7 × 1) KB
|
||||||
|
C→B 平均消息流量 ≈ λ × (0.7 × 4) KB
|
||||||
```
|
```
|
||||||
|
|
||||||
例如 $\lambda_{ext}=100$ 条/秒时：
|
例如 $\lambda_{ext}=100$ 条/秒时：
|
||||||
|
|
||||||
```text
|
```text
|
||||||
A-B ≈ 100 × 5 KB = 500 KB/秒
|
{A,B} 双向汇总 ≈ 100 × 5 KB = 500 KB/秒
|
||||||
B-C ≈ 100 × 0.7 × 5 KB = 350 KB/秒
|
{B,C} 双向汇总 ≈ 100 × 0.7 × 5 KB = 350 KB/秒
|
||||||
```
|
```
|
||||||
|
|
||||||
若考虑超时重试，则还需要乘以平均请求次数。若单次请求超时概率为 (p_{timeout})，且允许无限重试的理论平均请求次数为：
|
若考虑超时重试，则还需要乘以平均请求次数。若单次请求超时概率为 (p_{timeout})，且允许无限重试的理论平均请求次数为：
|
||||||
@@ -1451,16 +1581,17 @@ B-C ≈ 100 × 0.7 × 5 KB = 350 KB/秒
|
|||||||
|
|
||||||
## 11. 最终模型定义
|
## 11. 最终模型定义
|
||||||
|
|
||||||
一个消息或消息处理事件在时刻 $t$ 的完整状态可以表示为：
|
一个消息处理上下文或消息处理事件在时刻 $t$ 的完整状态可以表示为：
|
||||||
|
|
||||||
$$
|
$$
|
||||||
s_m(t)=(v,q_v,x_v,z_v,c,m,path,h,Q_v(t),r_v,\tau_v,{retry\_count})
|
s_m(t)=(v,q_v,q_m,x_v,z_v,c,m,path,h,Q_v(t),r_v,\tau_v,{retry\_count})
|
||||||
$$
|
$$
|
||||||
|
|
||||||
其中：
|
其中：
|
||||||
|
|
||||||
- $v$：消息当前所在的节点；
|
- $v$：消息当前所在的节点；
|
||||||
- $q_v$：节点当前内部状态；
|
- $q_v$：节点级资源状态，如 `Idle`、`Busy`、`Saturated` 或 `Failed`；
|
||||||
|
- $q_m$：当前消息或任务处理上下文的内部状态；
|
||||||
- $x_v$：节点能力和资源配置；
|
- $x_v$：节点能力和资源配置；
|
||||||
- $z_v$：从节点局部拓扑 $G_v$ 中提取的特征；
|
- $z_v$：从节点局部拓扑 $G_v$ 中提取的特征；
|
||||||
- $c$：任务类型、任务阶段和其他业务上下文；
|
- $c$：任务类型、任务阶段和其他业务上下文；
|
||||||
@@ -1468,7 +1599,7 @@ $$
|
|||||||
- $path$：消息已经经过或计划经过的节点序列；
|
- $path$：消息已经经过或计划经过的节点序列；
|
||||||
- $h$：消息已经完成的跳数；
|
- $h$：消息已经完成的跳数；
|
||||||
- $Q_v(t)$：节点当前等待队列长度；
|
- $Q_v(t)$：节点当前等待队列长度；
|
||||||
- $r_v$：节点当前并发数、处理资源占用和可用容量；
|
- $r_v$：节点当前并发数、处理资源占用和可用容量。它决定新的消息处理上下文能否从 `Queue` 进入活跃状态；
|
||||||
- $\tau_v$：节点或消息相关的时间信息，包括状态持续时间和等待时间；
|
- $\tau_v$：节点或消息相关的时间信息，包括状态持续时间和等待时间；
|
||||||
- retry_count：当前消息已经重试的次数。
|
- retry_count：当前消息已经重试的次数。
|
||||||
|
|
||||||
@@ -1522,7 +1653,8 @@ $$
|
|||||||
| `priority` | 消息优先级 | 离散等级 | 系统配置或日志 | 决定队列调度顺序 |
|
| `priority` | 消息优先级 | 离散等级 | 系统配置或日志 | 决定队列调度顺序 |
|
||||||
| `path` | 消息计划经过或已经经过的节点序列 | 节点序列 | 路由规则或日志 | 确定并更新每一跳的链路流量 |
|
| `path` | 消息计划经过或已经经过的节点序列 | 节点序列 | 路由规则或日志 | 确定并更新每一跳的链路流量 |
|
||||||
| $h$ | 消息已经完成的跳数 | 非负整数 | 根据路径实时更新 | 表示消息传播位置 |
|
| $h$ | 消息已经完成的跳数 | 非负整数 | 根据路径实时更新 | 表示消息传播位置 |
|
||||||
| $q$ | 节点内部状态 | 状态枚举 | 状态机定义和日志 | 决定下一步行为 |
|
| $q_v$ | 节点级资源状态 | `Idle`、`Busy`、`Saturated`、`Failed` | 状态机定义和日志 | 判断节点整体资源状态 |
|
||||||
|
| $q_m$ | 消息或任务处理上下文状态 | 状态枚举 | 状态机定义和日志 | 决定当前消息下一步行为 |
|
||||||
| $Q_v(t)$ | 节点$v$ 在时刻 $t$ 的等待消息数量 | 条数 | 仿真实时维护 | 计算排队延迟和拥塞 |
|
| $Q_v(t)$ | 节点$v$ 在时刻 $t$ 的等待消息数量 | 条数 | 仿真实时维护 | 计算排队延迟和拥塞 |
|
||||||
| $Q_v^{max}$ | 节点队列容量上限 | 条数 | 系统配置或压测 | 判断丢弃、拒绝或限流 |
|
| $Q_v^{max}$ | 节点队列容量上限 | 条数 | 系统配置或压测 | 判断丢弃、拒绝或限流 |
|
||||||
| $\lambda$ | 一般消息到达率 | 条/秒 | 日志或场景配置 | 生成输入消息 |
|
| $\lambda$ | 一般消息到达率 | 条/秒 | 日志或场景配置 | 生成输入消息 |
|
||||||
|
|||||||
@@ -0,0 +1,126 @@
|
|||||||
|
# Agent 网络流量快速实验工程
|
||||||
|
|
||||||
|
本工程用于快速完成以下闭环：
|
||||||
|
|
||||||
|
1. 通过大模型生成任务行为画像（任务阶段、工具调用、拆分、消息大小、处理时间等）；
|
||||||
|
2. 将行为画像扩展为结构化 Agent 事件日志；
|
||||||
|
3. 从训练日志估计状态转移、持续时间、消息大小、超时和分支参数；
|
||||||
|
4. 使用双层状态机、节点队列和离散事件调度进行仿真；
|
||||||
|
5. 自动完成预测验证、基线对比、压力实验和重试放大实验；
|
||||||
|
6. 输出 CSV、JSON 和 PNG 图表。
|
||||||
|
|
||||||
|
## 快速开始
|
||||||
|
|
||||||
|
当前环境已具备主要依赖，直接运行：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd agent_traffic_experiments
|
||||||
|
python run_pipeline.py
|
||||||
|
```
|
||||||
|
|
||||||
|
默认使用 `local` 模式，不需要 API Key。结果写入 `outputs/latest/`。
|
||||||
|
|
||||||
|
## 使用大模型生成行为画像
|
||||||
|
|
||||||
|
本工程兼容 OpenAI 风格的 `/v1/chat/completions` 接口。先设置环境变量：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
$env:OPENAI_API_KEY="你的密钥"
|
||||||
|
$env:OPENAI_BASE_URL="https://api.openai.com/v1"
|
||||||
|
$env:OPENAI_MODEL="你要使用的模型名称"
|
||||||
|
python run_pipeline.py --generator llm
|
||||||
|
```
|
||||||
|
|
||||||
|
也可以使用其他兼容服务，只需修改 `OPENAI_BASE_URL` 和 `OPENAI_MODEL`。
|
||||||
|
|
||||||
|
大模型只负责生成少量、可解释的任务行为画像；本地生成器会校验画像并扩展成大量事件。这样比让大模型直接输出数万行日志更稳定、更便宜，也能保证消息 ID、父子关系、时间戳和流量统计一致。
|
||||||
|
|
||||||
|
如果 LLM 请求失败，程序默认自动回退到本地画像。使用 `--no-fallback` 可禁止回退。
|
||||||
|
|
||||||
|
## 常用命令
|
||||||
|
|
||||||
|
只生成数据：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
python run_pipeline.py --steps generate
|
||||||
|
```
|
||||||
|
|
||||||
|
生成数据、估参并运行实验：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
python run_pipeline.py --steps generate,estimate,experiment
|
||||||
|
```
|
||||||
|
|
||||||
|
使用已有数据重新实验：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
python run_pipeline.py --steps estimate,experiment
|
||||||
|
```
|
||||||
|
|
||||||
|
指定配置和输出目录：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
python run_pipeline.py --config configs/default.yaml --output outputs/run_001
|
||||||
|
```
|
||||||
|
|
||||||
|
## 输出说明
|
||||||
|
|
||||||
|
```text
|
||||||
|
outputs/latest/
|
||||||
|
data/
|
||||||
|
behavior_profiles.json # LLM或本地生成的任务画像
|
||||||
|
train_events.csv # 参数估计数据
|
||||||
|
test_events.csv # 留出验证数据
|
||||||
|
task_truth.csv # 测试任务真实聚合值
|
||||||
|
parameters/
|
||||||
|
estimated_parameters.json # 从训练日志估计的模型参数
|
||||||
|
results/
|
||||||
|
validation_metrics.csv # 完整模型与基线误差
|
||||||
|
validation_predictions.csv # 逐场景预测值和真实值
|
||||||
|
stress_results.csv # 到达率压力实验
|
||||||
|
retry_results.csv # 超时与重试放大实验
|
||||||
|
summary.json # 实验摘要
|
||||||
|
figures/
|
||||||
|
prediction_vs_truth.png
|
||||||
|
model_comparison.png
|
||||||
|
stress_curves.png
|
||||||
|
retry_heatmap.png
|
||||||
|
```
|
||||||
|
|
||||||
|
## 实验内容
|
||||||
|
|
||||||
|
### E1 预测验证
|
||||||
|
|
||||||
|
训练集用于估计参数，测试集作为“模拟实测值”。完整状态机模型预测测试场景的内部字节数、任务时延、消息数和工具调用数。
|
||||||
|
|
||||||
|
### E2 基线对比
|
||||||
|
|
||||||
|
- `static_mean`：固定平均流量放大倍数；
|
||||||
|
- `no_context`：不区分任务类型和任务阶段；
|
||||||
|
- `full_model`：按任务画像和估计参数运行状态机仿真。
|
||||||
|
|
||||||
|
### E3 压力实验
|
||||||
|
|
||||||
|
逐渐提高外部到达率，观察吞吐量、平均/P95延迟、队列峰值、失败率和流量放大系数。
|
||||||
|
|
||||||
|
### E4 重试放大实验
|
||||||
|
|
||||||
|
改变超时概率和最大重试次数，观察任务成功率、平均重试数和内部流量放大系数。
|
||||||
|
|
||||||
|
## 数据字段
|
||||||
|
|
||||||
|
事件日志主要字段包括：
|
||||||
|
|
||||||
|
- `timestamp`、`task_id`、`message_id`、`parent_message_id`；
|
||||||
|
- `task_type`、`task_phase`、`source`、`destination`；
|
||||||
|
- `state_before`、`state_after`、`state_duration_ms`；
|
||||||
|
- `message_type`、`message_size_bytes`；
|
||||||
|
- `queue_length`、`success`、`retry_count`、`is_external`。
|
||||||
|
|
||||||
|
## 注意事项
|
||||||
|
|
||||||
|
- 当前数据是用于方法验证的模拟数据，不能冒充真实生产日志；
|
||||||
|
- LLM 画像应在报告中标注模型名称、生成时间和提示词版本；
|
||||||
|
- 正式参赛前，应尽量用少量真实 Agent 日志替换或校准模拟参数；
|
||||||
|
- 所有随机实验都由配置中的随机种子控制，便于复现。
|
||||||
|
|
||||||
Binary file not shown.
@@ -0,0 +1,37 @@
|
|||||||
|
seed: 20260813
|
||||||
|
|
||||||
|
generation:
|
||||||
|
train_tasks_per_type: 260
|
||||||
|
test_tasks_per_type: 100
|
||||||
|
task_types:
|
||||||
|
- simple_qa
|
||||||
|
- tool_research
|
||||||
|
- collaborative_analysis
|
||||||
|
llm:
|
||||||
|
temperature: 0.5
|
||||||
|
timeout_seconds: 90
|
||||||
|
max_tokens: 3500
|
||||||
|
|
||||||
|
network:
|
||||||
|
agent_count: 8
|
||||||
|
tool_count: 3
|
||||||
|
coordinator: agent_0
|
||||||
|
default_link_bandwidth_mbps: 20
|
||||||
|
default_link_delay_ms: 8
|
||||||
|
max_concurrency: 4
|
||||||
|
queue_capacity: 500
|
||||||
|
|
||||||
|
simulation:
|
||||||
|
replications: 16
|
||||||
|
timeout_ms: 2200
|
||||||
|
max_retries: 2
|
||||||
|
retry_backoff_ms: 180
|
||||||
|
|
||||||
|
experiments:
|
||||||
|
validation_tasks_per_scenario: 120
|
||||||
|
arrival_rates_per_second: [0.5, 1, 2, 4, 6, 8, 10, 12]
|
||||||
|
stress_duration_seconds: 180
|
||||||
|
stress_warmup_seconds: 20
|
||||||
|
timeout_probabilities: [0.0, 0.03, 0.08, 0.15, 0.25]
|
||||||
|
retry_limits: [0, 1, 2, 3, 5]
|
||||||
|
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
seed: 20260813
|
||||||
|
|
||||||
|
generation:
|
||||||
|
train_tasks_per_type: 260
|
||||||
|
test_tasks_per_type: 100
|
||||||
|
task_types:
|
||||||
|
- simple_qa
|
||||||
|
- tool_research
|
||||||
|
- collaborative_analysis
|
||||||
|
llm:
|
||||||
|
temperature: 0.5
|
||||||
|
timeout_seconds: 90
|
||||||
|
max_tokens: 3500
|
||||||
|
|
||||||
|
network:
|
||||||
|
agent_count: 8
|
||||||
|
tool_count: 3
|
||||||
|
coordinator: agent_0
|
||||||
|
default_link_bandwidth_mbps: 20
|
||||||
|
default_link_delay_ms: 8
|
||||||
|
max_concurrency: 4
|
||||||
|
queue_capacity: 500
|
||||||
|
|
||||||
|
simulation:
|
||||||
|
replications: 16
|
||||||
|
timeout_ms: 2200
|
||||||
|
max_retries: 2
|
||||||
|
retry_backoff_ms: 180
|
||||||
|
|
||||||
|
experiments:
|
||||||
|
validation_tasks_per_scenario: 120
|
||||||
|
arrival_rates_per_second: [0.5, 1, 2, 4, 6, 8, 10, 12]
|
||||||
|
stress_duration_seconds: 180
|
||||||
|
stress_warmup_seconds: 20
|
||||||
|
timeout_probabilities: [0.0, 0.03, 0.08, 0.15, 0.25]
|
||||||
|
retry_limits: [0, 1, 2, 3, 5]
|
||||||
|
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"generator": "local_builtin"
|
||||||
|
},
|
||||||
|
"profiles": [
|
||||||
|
{
|
||||||
|
"task_type": "simple_qa",
|
||||||
|
"description": "单节点即可完成的简短问答，少量情况下调用工具。",
|
||||||
|
"phase": "answering",
|
||||||
|
"tool_probability": 0.1,
|
||||||
|
"split_probability": 0.05,
|
||||||
|
"mean_subtasks": 1.2,
|
||||||
|
"timeout_probability": 0.02,
|
||||||
|
"failure_probability": 0.01,
|
||||||
|
"think_time_ms_mean": 420.0,
|
||||||
|
"think_time_cv": 0.55,
|
||||||
|
"request_size_bytes_mean": 1800.0,
|
||||||
|
"response_size_bytes_mean": 2600.0,
|
||||||
|
"message_size_cv": 0.45
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"task_type": "tool_research",
|
||||||
|
"description": "需要搜索、数据库或工具结果的研究任务。",
|
||||||
|
"phase": "evidence_collection",
|
||||||
|
"tool_probability": 0.78,
|
||||||
|
"split_probability": 0.2,
|
||||||
|
"mean_subtasks": 1.8,
|
||||||
|
"timeout_probability": 0.07,
|
||||||
|
"failure_probability": 0.025,
|
||||||
|
"think_time_ms_mean": 900.0,
|
||||||
|
"think_time_cv": 0.75,
|
||||||
|
"request_size_bytes_mean": 3200.0,
|
||||||
|
"response_size_bytes_mean": 8500.0,
|
||||||
|
"message_size_cv": 0.7
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"task_type": "collaborative_analysis",
|
||||||
|
"description": "协调多个执行 Agent 并汇总结果的复杂分析任务。",
|
||||||
|
"phase": "multi_agent_synthesis",
|
||||||
|
"tool_probability": 0.48,
|
||||||
|
"split_probability": 0.82,
|
||||||
|
"mean_subtasks": 3.4,
|
||||||
|
"timeout_probability": 0.09,
|
||||||
|
"failure_probability": 0.035,
|
||||||
|
"think_time_ms_mean": 1450.0,
|
||||||
|
"think_time_cv": 0.85,
|
||||||
|
"request_size_bytes_mean": 5200.0,
|
||||||
|
"response_size_bytes_mean": 11800.0,
|
||||||
|
"message_size_cv": 0.8
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,301 @@
|
|||||||
|
task_id,task_type,task_phase,latency_ms,internal_bytes,input_bytes,amplification,message_count,tool_calls,retries,success
|
||||||
|
collaborative_analysis_00000,collaborative_analysis,multi_agent_synthesis,6111.839000000004,62448,1205,51.824066390041494,7,1,0,True
|
||||||
|
collaborative_analysis_00001,collaborative_analysis,multi_agent_synthesis,7858.875999999981,57236,2452,23.34257748776509,7,0,0,True
|
||||||
|
collaborative_analysis_00002,collaborative_analysis,multi_agent_synthesis,3745.010999999977,36184,6916,5.231925968768074,5,1,0,True
|
||||||
|
collaborative_analysis_00003,collaborative_analysis,multi_agent_synthesis,3370.6700000000183,15600,1146,13.612565445026178,3,0,0,True
|
||||||
|
collaborative_analysis_00004,collaborative_analysis,multi_agent_synthesis,3970.753000000002,26927,5695,4.728182616330114,5,1,0,True
|
||||||
|
collaborative_analysis_00005,collaborative_analysis,multi_agent_synthesis,3692.181000000005,9783,2845,3.438664323374341,3,0,0,True
|
||||||
|
collaborative_analysis_00006,collaborative_analysis,multi_agent_synthesis,12027.614,101344,5088,19.91823899371069,9,1,0,True
|
||||||
|
collaborative_analysis_00007,collaborative_analysis,multi_agent_synthesis,3174.7689999999975,13427,10177,1.3193475483934363,3,0,0,True
|
||||||
|
collaborative_analysis_00008,collaborative_analysis,multi_agent_synthesis,4553.435000000008,106516,5924,17.98041863605672,13,2,0,True
|
||||||
|
collaborative_analysis_00009,collaborative_analysis,multi_agent_synthesis,20393.187000000013,136393,8870,15.376888387824126,19,3,0,True
|
||||||
|
collaborative_analysis_00010,collaborative_analysis,multi_agent_synthesis,9716.779000000002,43830,1420,30.866197183098592,9,2,0,True
|
||||||
|
collaborative_analysis_00011,collaborative_analysis,multi_agent_synthesis,13658.126999999979,113877,6803,16.73923269145965,13,2,0,True
|
||||||
|
collaborative_analysis_00012,collaborative_analysis,multi_agent_synthesis,5092.640999999986,138420,4391,31.523570940560237,13,2,0,True
|
||||||
|
collaborative_analysis_00013,collaborative_analysis,multi_agent_synthesis,2393.601999999987,13198,8494,1.5538026842477042,3,0,0,True
|
||||||
|
collaborative_analysis_00014,collaborative_analysis,multi_agent_synthesis,7330.83400000001,65750,876,75.05707762557077,11,2,0,True
|
||||||
|
collaborative_analysis_00015,collaborative_analysis,multi_agent_synthesis,16492.502,180400,2157,83.63467779323133,23,3,0,True
|
||||||
|
collaborative_analysis_00016,collaborative_analysis,multi_agent_synthesis,16875.831000000006,205302,2605,78.81074856046065,19,3,0,True
|
||||||
|
collaborative_analysis_00017,collaborative_analysis,multi_agent_synthesis,9850.209000000006,147782,1988,74.33702213279678,19,3,0,True
|
||||||
|
collaborative_analysis_00018,collaborative_analysis,multi_agent_synthesis,10007.738999999987,126126,4218,29.90184921763869,15,2,0,True
|
||||||
|
collaborative_analysis_00019,collaborative_analysis,multi_agent_synthesis,11525.198999999986,106483,1192,89.33137583892618,7,1,0,True
|
||||||
|
collaborative_analysis_00020,collaborative_analysis,multi_agent_synthesis,5159.906000000006,59557,2210,26.94886877828054,9,2,0,True
|
||||||
|
collaborative_analysis_00021,collaborative_analysis,multi_agent_synthesis,4791.642999999994,57467,18445,3.1155868799132556,5,0,0,True
|
||||||
|
collaborative_analysis_00022,collaborative_analysis,multi_agent_synthesis,11351.583000000006,75128,3315,22.663046757164405,9,1,0,True
|
||||||
|
collaborative_analysis_00023,collaborative_analysis,multi_agent_synthesis,8277.253999999999,39603,5217,7.591144335825187,5,1,0,True
|
||||||
|
collaborative_analysis_00024,collaborative_analysis,multi_agent_synthesis,19919.341000000004,167247,1166,143.43653516295026,23,5,0,True
|
||||||
|
collaborative_analysis_00025,collaborative_analysis,multi_agent_synthesis,9099.73500000001,69536,6843,10.161625018266841,11,1,0,True
|
||||||
|
collaborative_analysis_00026,collaborative_analysis,multi_agent_synthesis,3098.5450000000014,55189,7507,7.351671773011856,11,2,0,True
|
||||||
|
collaborative_analysis_00027,collaborative_analysis,multi_agent_synthesis,6933.150000000012,128239,5952,21.545530913978496,13,3,0,True
|
||||||
|
collaborative_analysis_00028,collaborative_analysis,multi_agent_synthesis,14684.41899999999,164414,1446,113.70262793914246,18,5,1,True
|
||||||
|
collaborative_analysis_00029,collaborative_analysis,multi_agent_synthesis,22606.593000000004,130445,2700,48.31296296296296,13,3,2,True
|
||||||
|
collaborative_analysis_00030,collaborative_analysis,multi_agent_synthesis,7566.215999999997,53530,2840,18.848591549295776,7,0,0,True
|
||||||
|
collaborative_analysis_00031,collaborative_analysis,multi_agent_synthesis,9549.07,182717,560,326.28035714285716,11,2,0,True
|
||||||
|
collaborative_analysis_00032,collaborative_analysis,multi_agent_synthesis,16413.354,140512,4336,32.40590405904059,13,1,0,True
|
||||||
|
collaborative_analysis_00033,collaborative_analysis,multi_agent_synthesis,3107.8390000000127,25062,1526,16.42332896461337,3,0,0,True
|
||||||
|
collaborative_analysis_00034,collaborative_analysis,multi_agent_synthesis,2382.689999999997,37813,4410,8.57437641723356,5,1,0,True
|
||||||
|
collaborative_analysis_00035,collaborative_analysis,multi_agent_synthesis,6573.08900000001,42752,1231,34.72948822095857,9,2,0,True
|
||||||
|
collaborative_analysis_00036,collaborative_analysis,multi_agent_synthesis,14503.11400000001,179234,7724,23.20481615743138,23,6,1,True
|
||||||
|
collaborative_analysis_00037,collaborative_analysis,multi_agent_synthesis,6998.415999999992,40737,4407,9.243703199455412,5,1,0,True
|
||||||
|
collaborative_analysis_00038,collaborative_analysis,multi_agent_synthesis,9218.387000000006,76646,5615,13.650222617987533,12,3,1,True
|
||||||
|
collaborative_analysis_00039,collaborative_analysis,multi_agent_synthesis,14171.648999999974,82798,3958,20.919151086407275,13,2,0,True
|
||||||
|
collaborative_analysis_00040,collaborative_analysis,multi_agent_synthesis,16203.926999999992,192384,4272,45.03370786516854,19,2,0,True
|
||||||
|
collaborative_analysis_00041,collaborative_analysis,multi_agent_synthesis,8077.832999999999,86864,14703,5.907909950350269,9,1,0,True
|
||||||
|
collaborative_analysis_00042,collaborative_analysis,multi_agent_synthesis,11980.147000000017,175669,4129,42.545168321627514,17,2,0,True
|
||||||
|
collaborative_analysis_00043,collaborative_analysis,multi_agent_synthesis,8383.41299999999,86553,4556,18.99758560140474,11,1,0,True
|
||||||
|
collaborative_analysis_00044,collaborative_analysis,multi_agent_synthesis,3781.3569999999854,21537,2248,9.580516014234876,5,1,0,True
|
||||||
|
collaborative_analysis_00045,collaborative_analysis,multi_agent_synthesis,2871.480999999989,29178,3997,7.299974981235927,5,1,0,True
|
||||||
|
collaborative_analysis_00046,collaborative_analysis,multi_agent_synthesis,1987.737999999979,21361,8073,2.6459804285891244,3,0,0,True
|
||||||
|
collaborative_analysis_00047,collaborative_analysis,multi_agent_synthesis,10075.866999999987,219312,9941,22.061362036012472,17,3,0,True
|
||||||
|
collaborative_analysis_00048,collaborative_analysis,multi_agent_synthesis,6179.957999999999,70354,2470,28.4834008097166,5,1,0,True
|
||||||
|
collaborative_analysis_00049,collaborative_analysis,multi_agent_synthesis,3145.8080000000164,28609,4669,6.127436281859071,3,0,0,True
|
||||||
|
collaborative_analysis_00050,collaborative_analysis,multi_agent_synthesis,6249.4649999999865,28317,882,32.105442176870746,7,1,0,True
|
||||||
|
collaborative_analysis_00051,collaborative_analysis,multi_agent_synthesis,11702.425000000005,187198,1275,146.82196078431372,17,2,0,True
|
||||||
|
collaborative_analysis_00052,collaborative_analysis,multi_agent_synthesis,10426.145999999988,122179,1403,87.08410548823949,13,2,0,True
|
||||||
|
collaborative_analysis_00053,collaborative_analysis,multi_agent_synthesis,3776.5890000000013,10701,3073,3.482264887731858,3,0,0,True
|
||||||
|
collaborative_analysis_00054,collaborative_analysis,multi_agent_synthesis,6411.059999999992,122716,3644,33.6761800219539,13,3,0,True
|
||||||
|
collaborative_analysis_00055,collaborative_analysis,multi_agent_synthesis,9190.782999999981,87827,8202,10.707998049256279,13,1,0,True
|
||||||
|
collaborative_analysis_00056,collaborative_analysis,multi_agent_synthesis,26072.79299999999,196393,2732,71.88616398243046,22,5,1,True
|
||||||
|
collaborative_analysis_00057,collaborative_analysis,multi_agent_synthesis,6656.256000000013,33081,7347,4.502654144548796,7,1,0,True
|
||||||
|
collaborative_analysis_00058,collaborative_analysis,multi_agent_synthesis,5274.61199999999,58229,9303,6.259163710630979,8,2,1,True
|
||||||
|
collaborative_analysis_00059,collaborative_analysis,multi_agent_synthesis,5612.335999999999,17516,7543,2.3221529895267135,3,0,0,True
|
||||||
|
collaborative_analysis_00060,collaborative_analysis,multi_agent_synthesis,7323.184999999995,88159,6584,13.389884568651276,5,1,0,True
|
||||||
|
collaborative_analysis_00061,collaborative_analysis,multi_agent_synthesis,13041.410999999982,39582,4651,8.510427864975274,5,0,0,True
|
||||||
|
collaborative_analysis_00062,collaborative_analysis,multi_agent_synthesis,5737.938000000014,55676,4041,13.777777777777779,11,2,0,True
|
||||||
|
collaborative_analysis_00063,collaborative_analysis,multi_agent_synthesis,13871.848999999998,111676,5758,19.39492879472039,18,5,1,True
|
||||||
|
collaborative_analysis_00064,collaborative_analysis,multi_agent_synthesis,14159.884000000005,93620,743,126.00269179004037,11,2,0,True
|
||||||
|
collaborative_analysis_00065,collaborative_analysis,multi_agent_synthesis,2143.2680000000064,44223,2414,18.319386909693456,5,0,0,True
|
||||||
|
collaborative_analysis_00066,collaborative_analysis,multi_agent_synthesis,9103.234999999984,120161,2301,52.22120817036071,11,0,0,True
|
||||||
|
collaborative_analysis_00067,collaborative_analysis,multi_agent_synthesis,7975.786999999997,51818,2211,23.43645409317051,9,2,0,True
|
||||||
|
collaborative_analysis_00068,collaborative_analysis,multi_agent_synthesis,10306.556999999997,143550,1168,122.90239726027397,15,3,0,True
|
||||||
|
collaborative_analysis_00069,collaborative_analysis,multi_agent_synthesis,10491.949999999975,88142,1034,85.24371373307544,13,3,0,True
|
||||||
|
collaborative_analysis_00070,collaborative_analysis,multi_agent_synthesis,5671.043999999994,92836,4373,21.229361994054425,7,1,0,True
|
||||||
|
collaborative_analysis_00071,collaborative_analysis,multi_agent_synthesis,2233.918000000017,35792,908,39.418502202643175,5,1,0,True
|
||||||
|
collaborative_analysis_00072,collaborative_analysis,multi_agent_synthesis,2779.2799999999997,19132,1140,16.782456140350877,3,0,0,True
|
||||||
|
collaborative_analysis_00073,collaborative_analysis,multi_agent_synthesis,5933.890999999989,53137,2409,22.057700290577003,7,1,0,True
|
||||||
|
collaborative_analysis_00074,collaborative_analysis,multi_agent_synthesis,11494.603000000012,198018,8671,22.836812363049244,15,5,1,True
|
||||||
|
collaborative_analysis_00075,collaborative_analysis,multi_agent_synthesis,3800.8010000000068,92249,1891,48.78318350079323,7,0,0,True
|
||||||
|
collaborative_analysis_00076,collaborative_analysis,multi_agent_synthesis,15271.408000000009,110064,4836,22.759305210918114,13,4,2,True
|
||||||
|
collaborative_analysis_00077,collaborative_analysis,multi_agent_synthesis,3524.1379999999936,49633,2359,21.03984739296312,7,0,0,True
|
||||||
|
collaborative_analysis_00078,collaborative_analysis,multi_agent_synthesis,1085.5840000000114,34459,1481,23.267386900742743,3,0,0,True
|
||||||
|
collaborative_analysis_00079,collaborative_analysis,multi_agent_synthesis,3396.9510000000014,7311,4899,1.492345376607471,3,0,0,True
|
||||||
|
collaborative_analysis_00080,collaborative_analysis,multi_agent_synthesis,7375.2580000000025,40747,2612,15.599923430321592,10,3,1,True
|
||||||
|
collaborative_analysis_00081,collaborative_analysis,multi_agent_synthesis,11392.599000000018,125175,5129,24.405342171963344,17,5,1,True
|
||||||
|
collaborative_analysis_00082,collaborative_analysis,multi_agent_synthesis,4221.8009999999995,46903,1193,39.3151718357083,3,0,0,True
|
||||||
|
collaborative_analysis_00083,collaborative_analysis,multi_agent_synthesis,4469.9559999999965,34392,7835,4.389534141671985,5,0,0,True
|
||||||
|
collaborative_analysis_00084,collaborative_analysis,multi_agent_synthesis,14913.923000000012,148982,4218,35.32053105737316,17,5,2,False
|
||||||
|
collaborative_analysis_00085,collaborative_analysis,multi_agent_synthesis,3174.0640000000158,26574,2863,9.281872162067762,3,0,0,True
|
||||||
|
collaborative_analysis_00086,collaborative_analysis,multi_agent_synthesis,6104.687000000013,29578,2252,13.134103019538188,6,2,1,True
|
||||||
|
collaborative_analysis_00087,collaborative_analysis,multi_agent_synthesis,3079.0619999999935,23392,7001,3.3412369661476933,3,0,0,True
|
||||||
|
collaborative_analysis_00088,collaborative_analysis,multi_agent_synthesis,15710.582000000017,123812,1087,113.90248390064397,17,3,0,True
|
||||||
|
collaborative_analysis_00089,collaborative_analysis,multi_agent_synthesis,9594.03499999999,123841,7467,16.585107807687155,15,3,0,True
|
||||||
|
collaborative_analysis_00090,collaborative_analysis,multi_agent_synthesis,8018.177000000009,35223,2432,14.483141447368421,7,1,0,True
|
||||||
|
collaborative_analysis_00091,collaborative_analysis,multi_agent_synthesis,8070.577999999983,85639,1800,47.577222222222225,9,1,0,True
|
||||||
|
collaborative_analysis_00092,collaborative_analysis,multi_agent_synthesis,6954.7670000000035,30543,2606,11.720260936300845,8,2,1,True
|
||||||
|
collaborative_analysis_00093,collaborative_analysis,multi_agent_synthesis,2334.4150000000072,9761,1259,7.75297855440826,3,0,0,True
|
||||||
|
collaborative_analysis_00094,collaborative_analysis,multi_agent_synthesis,3988.2240000000024,53918,4052,13.306515301085884,7,0,0,True
|
||||||
|
collaborative_analysis_00095,collaborative_analysis,multi_agent_synthesis,6538.421999999997,67526,4362,15.480513525905549,7,1,0,True
|
||||||
|
collaborative_analysis_00096,collaborative_analysis,multi_agent_synthesis,7050.077000000016,89764,5545,16.188277727682596,9,0,0,True
|
||||||
|
collaborative_analysis_00097,collaborative_analysis,multi_agent_synthesis,14613.135,138234,1905,72.56377952755905,18,4,1,True
|
||||||
|
collaborative_analysis_00098,collaborative_analysis,multi_agent_synthesis,5859.162999999995,34186,15338,2.2288433954883295,6,2,1,True
|
||||||
|
collaborative_analysis_00099,collaborative_analysis,multi_agent_synthesis,17759.783999999996,169187,11129,15.202354209722348,21,5,0,True
|
||||||
|
simple_qa_00000,simple_qa,answering,855.1170000000001,3448,2896,1.1906077348066297,3,0,0,True
|
||||||
|
simple_qa_00001,simple_qa,answering,1403.896,4962,1531,3.24101894186806,3,0,0,True
|
||||||
|
simple_qa_00002,simple_qa,answering,1171.8220000000001,4350,787,5.527318932655654,3,0,0,True
|
||||||
|
simple_qa_00003,simple_qa,answering,1504.4239999999998,2494,375,6.650666666666667,3,0,0,True
|
||||||
|
simple_qa_00004,simple_qa,answering,1963.5280000000002,5963,591,10.089678510998308,5,1,0,True
|
||||||
|
simple_qa_00005,simple_qa,answering,1452.445,5943,2101,2.8286530223703,3,0,0,True
|
||||||
|
simple_qa_00006,simple_qa,answering,825.0729999999998,3757,1209,3.10752688172043,3,0,0,True
|
||||||
|
simple_qa_00007,simple_qa,answering,926.895,3668,2132,1.7204502814258913,3,0,0,True
|
||||||
|
simple_qa_00008,simple_qa,answering,1005.903,4694,1529,3.0699803793328972,3,0,0,True
|
||||||
|
simple_qa_00009,simple_qa,answering,1038.875,5016,1632,3.073529411764706,3,0,0,True
|
||||||
|
simple_qa_00010,simple_qa,answering,659.4789999999992,9518,859,11.080325960419092,3,0,0,True
|
||||||
|
simple_qa_00011,simple_qa,answering,1125.273,3965,1038,3.8198458574181116,3,0,0,True
|
||||||
|
simple_qa_00012,simple_qa,answering,1077.1380000000015,5386,1219,4.418375717801476,3,0,0,True
|
||||||
|
simple_qa_00013,simple_qa,answering,1004.5659999999988,5660,889,6.366704161979753,3,0,0,True
|
||||||
|
simple_qa_00014,simple_qa,answering,996.1830000000002,4420,1161,3.8070628768303187,3,0,0,True
|
||||||
|
simple_qa_00015,simple_qa,answering,860.6720000000009,4252,1484,2.8652291105121295,3,0,0,True
|
||||||
|
simple_qa_00016,simple_qa,answering,781.1749999999993,6704,1189,5.638351555929352,3,0,0,True
|
||||||
|
simple_qa_00017,simple_qa,answering,700.6979999999992,4185,1526,2.7424639580602883,3,0,0,True
|
||||||
|
simple_qa_00018,simple_qa,answering,1115.2699999999988,5439,1075,5.05953488372093,3,0,0,True
|
||||||
|
simple_qa_00019,simple_qa,answering,850.3690000000006,3539,1636,2.16320293398533,3,0,0,True
|
||||||
|
simple_qa_00020,simple_qa,answering,823.193999999999,5658,1063,5.322671683913453,3,0,0,True
|
||||||
|
simple_qa_00021,simple_qa,answering,930.963000000002,6726,2009,3.3479342956694875,3,0,0,True
|
||||||
|
simple_qa_00022,simple_qa,answering,742.083000000001,5048,1822,2.770581778265642,3,0,0,True
|
||||||
|
simple_qa_00023,simple_qa,answering,1172.7469999999976,3674,1567,2.3446075303126994,3,0,0,True
|
||||||
|
simple_qa_00024,simple_qa,answering,604.9790000000002,7257,1006,7.213717693836978,3,0,0,True
|
||||||
|
simple_qa_00025,simple_qa,answering,1219.6170000000031,4464,2084,2.1420345489443378,3,0,0,True
|
||||||
|
simple_qa_00026,simple_qa,answering,753.5990000000013,4068,1755,2.317948717948718,3,0,0,True
|
||||||
|
simple_qa_00027,simple_qa,answering,791.2379999999998,5056,1373,3.6824471959213403,3,0,0,True
|
||||||
|
simple_qa_00028,simple_qa,answering,827.5049999999986,3795,801,4.737827715355805,3,0,0,True
|
||||||
|
simple_qa_00029,simple_qa,answering,677.6099999999979,4655,2300,2.023913043478261,3,0,0,True
|
||||||
|
simple_qa_00030,simple_qa,answering,791.8140000000022,4267,720,5.926388888888889,3,0,0,True
|
||||||
|
simple_qa_00031,simple_qa,answering,1215.0280000000002,7292,3124,2.3341869398207424,5,1,0,True
|
||||||
|
simple_qa_00032,simple_qa,answering,1804.2910000000027,4607,2837,1.6238984843144166,3,0,0,True
|
||||||
|
simple_qa_00033,simple_qa,answering,844.0370000000001,3221,1756,1.8342824601366743,3,0,0,True
|
||||||
|
simple_qa_00034,simple_qa,answering,1057.2420000000022,5454,1504,3.6263297872340425,3,0,0,True
|
||||||
|
simple_qa_00035,simple_qa,answering,1576.5220000000006,5750,1500,3.8333333333333335,3,0,0,True
|
||||||
|
simple_qa_00036,simple_qa,answering,1026.4500000000005,7409,4583,1.616626663757364,3,0,0,True
|
||||||
|
simple_qa_00037,simple_qa,answering,716.7670000000008,5122,1777,2.8823860438942037,3,0,0,True
|
||||||
|
simple_qa_00038,simple_qa,answering,982.6349999999984,7826,1145,6.834934497816594,3,0,0,True
|
||||||
|
simple_qa_00039,simple_qa,answering,701.6949999999973,5078,670,7.57910447761194,3,0,0,True
|
||||||
|
simple_qa_00040,simple_qa,answering,1318.7110000000005,5824,2164,2.6913123844731976,3,0,0,True
|
||||||
|
simple_qa_00041,simple_qa,answering,839.9459999999976,6330,1513,4.183740912095175,3,0,0,True
|
||||||
|
simple_qa_00042,simple_qa,answering,1454.4789999999991,5313,1358,3.9123711340206184,3,0,0,True
|
||||||
|
simple_qa_00043,simple_qa,answering,1366.8249999999987,5530,733,7.544338335607094,3,0,0,True
|
||||||
|
simple_qa_00044,simple_qa,answering,1134.8300000000008,5574,919,6.065288356909685,3,0,0,True
|
||||||
|
simple_qa_00045,simple_qa,answering,1290.6880000000028,8607,1321,6.51551854655564,5,1,0,True
|
||||||
|
simple_qa_00046,simple_qa,answering,887.5910000000005,5342,1220,4.378688524590164,3,0,0,True
|
||||||
|
simple_qa_00047,simple_qa,answering,1500.1400000000017,11544,1496,7.716577540106952,5,0,0,True
|
||||||
|
simple_qa_00048,simple_qa,answering,590.2030000000025,3429,1270,2.7,3,0,0,True
|
||||||
|
simple_qa_00049,simple_qa,answering,1328.8580000000038,3926,1557,2.5215157353885678,3,0,0,True
|
||||||
|
simple_qa_00050,simple_qa,answering,895.7130000000006,4332,1319,3.284306292645944,3,0,0,True
|
||||||
|
simple_qa_00051,simple_qa,answering,754.6930000000032,6203,1342,4.62220566318927,3,0,0,True
|
||||||
|
simple_qa_00052,simple_qa,answering,1541.4049999999975,5071,1117,4.539838854073411,3,0,0,True
|
||||||
|
simple_qa_00053,simple_qa,answering,1177.0819999999985,8016,1230,6.517073170731707,5,1,0,True
|
||||||
|
simple_qa_00054,simple_qa,answering,498.7519999999961,5493,391,14.048593350383632,3,0,0,True
|
||||||
|
simple_qa_00055,simple_qa,answering,1162.9900000000007,7303,1651,4.423379769836463,3,0,0,True
|
||||||
|
simple_qa_00056,simple_qa,answering,674.510000000005,2802,669,4.188340807174888,3,0,0,True
|
||||||
|
simple_qa_00057,simple_qa,answering,2256.7659999999987,4338,969,4.476780185758514,3,0,0,True
|
||||||
|
simple_qa_00058,simple_qa,answering,1575.8580000000038,10725,3214,3.336963285625389,5,1,0,True
|
||||||
|
simple_qa_00059,simple_qa,answering,1815.6789999999958,5774,1265,4.564426877470356,3,0,0,True
|
||||||
|
simple_qa_00060,simple_qa,answering,686.0010000000045,5624,923,6.0931744312026,3,0,0,True
|
||||||
|
simple_qa_00061,simple_qa,answering,1254.2200000000037,6000,887,6.764374295377678,3,0,0,True
|
||||||
|
simple_qa_00062,simple_qa,answering,945.3009999999936,7475,564,13.25354609929078,3,0,0,True
|
||||||
|
simple_qa_00063,simple_qa,answering,841.0459999999987,3683,751,4.9041278295605855,3,0,0,True
|
||||||
|
simple_qa_00064,simple_qa,answering,1241.6660000000022,9729,904,10.76216814159292,5,1,0,True
|
||||||
|
simple_qa_00065,simple_qa,answering,1516.855999999997,6629,1109,5.977457168620378,5,1,0,True
|
||||||
|
simple_qa_00066,simple_qa,answering,1183.698999999997,3552,1627,2.183159188690842,3,0,0,True
|
||||||
|
simple_qa_00067,simple_qa,answering,995.9650000000054,8377,2106,3.977682811016144,3,0,0,True
|
||||||
|
simple_qa_00068,simple_qa,answering,959.3050000000005,4731,1465,3.2293515358361775,3,0,0,True
|
||||||
|
simple_qa_00069,simple_qa,answering,1188.1830000000023,3612,1113,3.2452830188679247,3,0,0,True
|
||||||
|
simple_qa_00070,simple_qa,answering,1491.27,4065,761,5.341655716162943,3,0,0,True
|
||||||
|
simple_qa_00071,simple_qa,answering,846.0699999999974,2898,1350,2.1466666666666665,3,0,0,True
|
||||||
|
simple_qa_00072,simple_qa,answering,1207.8729999999994,3417,747,4.57429718875502,3,0,0,True
|
||||||
|
simple_qa_00073,simple_qa,answering,1940.8279999999963,7028,3264,2.153186274509804,5,1,0,True
|
||||||
|
simple_qa_00074,simple_qa,answering,1482.2769999999964,5076,1081,4.695652173913044,3,0,0,True
|
||||||
|
simple_qa_00075,simple_qa,answering,864.4090000000019,6816,897,7.59866220735786,3,0,0,True
|
||||||
|
simple_qa_00076,simple_qa,answering,1091.8100000000024,4841,1310,3.6954198473282442,3,0,0,True
|
||||||
|
simple_qa_00077,simple_qa,answering,1149.9900000000025,4287,737,5.8168249660786975,3,0,0,True
|
||||||
|
simple_qa_00078,simple_qa,answering,1253.6500000000003,6102,1037,5.884281581485053,3,0,0,True
|
||||||
|
simple_qa_00079,simple_qa,answering,2618.4089999999997,6982,680,10.26764705882353,5,1,0,True
|
||||||
|
simple_qa_00080,simple_qa,answering,838.6209999999962,5148,824,6.247572815533981,3,0,0,True
|
||||||
|
simple_qa_00081,simple_qa,answering,706.4880000000003,8520,643,13.250388802488336,3,0,0,True
|
||||||
|
simple_qa_00082,simple_qa,answering,1185.3399999999965,4372,474,9.223628691983123,3,0,0,True
|
||||||
|
simple_qa_00083,simple_qa,answering,1010.4350000000011,8796,1347,6.5300668151447665,3,0,0,True
|
||||||
|
simple_qa_00084,simple_qa,answering,652.358999999997,3394,1270,2.6724409448818895,3,0,0,True
|
||||||
|
simple_qa_00085,simple_qa,answering,1272.1099999999979,5066,1257,4.030230708035004,3,0,0,True
|
||||||
|
simple_qa_00086,simple_qa,answering,1247.2220000000007,7260,610,11.901639344262295,5,1,0,True
|
||||||
|
simple_qa_00087,simple_qa,answering,813.3010000000027,3990,2088,1.910919540229885,3,0,0,True
|
||||||
|
simple_qa_00088,simple_qa,answering,1376.7839999999935,2576,1292,1.9938080495356036,3,0,0,True
|
||||||
|
simple_qa_00089,simple_qa,answering,727.212999999999,5616,1291,4.350116189000775,3,0,0,True
|
||||||
|
simple_qa_00090,simple_qa,answering,1237.8660000000039,7402,851,8.698002350176264,3,0,0,True
|
||||||
|
simple_qa_00091,simple_qa,answering,757.8150000000007,9526,1514,6.291941875825628,5,1,0,True
|
||||||
|
simple_qa_00092,simple_qa,answering,2362.844000000003,4959,1026,4.833333333333333,3,0,0,True
|
||||||
|
simple_qa_00093,simple_qa,answering,1152.037,4596,920,4.995652173913044,3,0,0,True
|
||||||
|
simple_qa_00094,simple_qa,answering,1841.4469999999951,4404,1581,2.7855787476280836,3,0,0,True
|
||||||
|
simple_qa_00095,simple_qa,answering,1221.4370000000017,5886,1182,4.979695431472082,3,0,0,True
|
||||||
|
simple_qa_00096,simple_qa,answering,1167.4190000000024,5641,2854,1.9765241765942536,3,0,0,True
|
||||||
|
simple_qa_00097,simple_qa,answering,1575.9909999999948,4913,1439,3.414176511466296,3,0,0,True
|
||||||
|
simple_qa_00098,simple_qa,answering,1244.641999999999,8507,1101,7.72661217075386,3,0,0,True
|
||||||
|
simple_qa_00099,simple_qa,answering,1048.887999999998,4661,1839,2.5345296356715608,3,0,0,True
|
||||||
|
tool_research_00000,tool_research,evidence_collection,2194.9050000000057,24867,3202,7.766083697688944,5,1,0,True
|
||||||
|
tool_research_00001,tool_research,evidence_collection,3702.2710000000034,43758,1917,22.826291079812208,9,2,0,True
|
||||||
|
tool_research_00002,tool_research,evidence_collection,4914.04399999999,29867,1906,15.669989506820567,5,1,0,True
|
||||||
|
tool_research_00003,tool_research,evidence_collection,2975.851999999996,18814,4275,4.40093567251462,5,1,0,True
|
||||||
|
tool_research_00004,tool_research,evidence_collection,1859.4280000000012,17926,1263,14.193190815518607,3,0,0,True
|
||||||
|
tool_research_00005,tool_research,evidence_collection,1666.4189999999976,13686,8729,1.56787719097262,5,1,0,True
|
||||||
|
tool_research_00006,tool_research,evidence_collection,2696.655000000007,9331,1703,5.479154433352907,3,0,0,True
|
||||||
|
tool_research_00007,tool_research,evidence_collection,5450.783000000001,13849,1654,8.373035066505441,6,2,1,True
|
||||||
|
tool_research_00008,tool_research,evidence_collection,1531.513000000004,17238,3296,5.22997572815534,5,1,0,True
|
||||||
|
tool_research_00009,tool_research,evidence_collection,4081.8340000000007,13926,436,31.940366972477065,5,1,0,True
|
||||||
|
tool_research_00010,tool_research,evidence_collection,4327.119000000011,41953,2235,18.770917225950782,5,1,0,True
|
||||||
|
tool_research_00011,tool_research,evidence_collection,3781.2230000000113,15091,3794,3.977596204533474,5,1,0,True
|
||||||
|
tool_research_00012,tool_research,evidence_collection,1917.4310000000078,14626,1784,8.198430493273543,3,0,0,True
|
||||||
|
tool_research_00013,tool_research,evidence_collection,3997.425000000007,13485,1603,8.412351840299438,5,1,0,True
|
||||||
|
tool_research_00014,tool_research,evidence_collection,2441.9809999999984,36453,2047,17.80801172447484,5,1,0,True
|
||||||
|
tool_research_00015,tool_research,evidence_collection,3959.5280000000057,15873,1390,11.419424460431655,3,0,0,True
|
||||||
|
tool_research_00016,tool_research,evidence_collection,1198.003,13816,976,14.155737704918034,3,0,0,True
|
||||||
|
tool_research_00017,tool_research,evidence_collection,4552.255999999999,16333,2462,6.634037367993502,5,1,0,True
|
||||||
|
tool_research_00018,tool_research,evidence_collection,6950.037999999992,39260,5865,6.693947144075021,6,2,1,True
|
||||||
|
tool_research_00019,tool_research,evidence_collection,3067.522999999994,15009,2037,7.368188512518409,3,0,0,True
|
||||||
|
tool_research_00020,tool_research,evidence_collection,4835.037999999997,30787,1662,18.524067388688326,7,1,0,True
|
||||||
|
tool_research_00021,tool_research,evidence_collection,8310.215999999997,113647,4305,26.39883855981417,17,4,0,True
|
||||||
|
tool_research_00022,tool_research,evidence_collection,3200.504999999993,18179,1048,17.346374045801525,5,1,0,True
|
||||||
|
tool_research_00023,tool_research,evidence_collection,13355.828000000003,126809,1205,105.2356846473029,15,5,1,True
|
||||||
|
tool_research_00024,tool_research,evidence_collection,3643.411999999998,21779,2025,10.755061728395061,5,1,0,True
|
||||||
|
tool_research_00025,tool_research,evidence_collection,9670.201000000006,47663,1365,34.91794871794872,13,4,2,True
|
||||||
|
tool_research_00026,tool_research,evidence_collection,1291.9939999999883,12623,3997,3.1581185889417065,3,0,0,True
|
||||||
|
tool_research_00027,tool_research,evidence_collection,4673.997999999998,27499,1513,18.17514871116986,5,1,0,True
|
||||||
|
tool_research_00028,tool_research,evidence_collection,2161.7999999999993,32414,1401,23.13633119200571,5,1,0,True
|
||||||
|
tool_research_00029,tool_research,evidence_collection,5903.75499999999,21437,3963,5.40928589452435,5,1,0,True
|
||||||
|
tool_research_00030,tool_research,evidence_collection,4832.2890000000025,62170,2517,24.70003972983711,9,2,0,True
|
||||||
|
tool_research_00031,tool_research,evidence_collection,4330.880000000007,29507,2884,10.23127600554785,6,2,1,True
|
||||||
|
tool_research_00032,tool_research,evidence_collection,1570.5789999999952,18174,849,21.406360424028268,5,1,0,True
|
||||||
|
tool_research_00033,tool_research,evidence_collection,1797.4840000000113,34726,2226,15.600179694519317,5,1,0,True
|
||||||
|
tool_research_00034,tool_research,evidence_collection,3040.871999999993,19123,2094,9.13228271251194,5,1,0,True
|
||||||
|
tool_research_00035,tool_research,evidence_collection,2986.626000000001,24572,1303,18.858019953952418,5,1,0,True
|
||||||
|
tool_research_00036,tool_research,evidence_collection,3130.6490000000053,28061,1577,17.793912492073556,3,0,0,True
|
||||||
|
tool_research_00037,tool_research,evidence_collection,3123.7370000000055,22728,3665,6.201364256480218,5,1,0,True
|
||||||
|
tool_research_00038,tool_research,evidence_collection,1708.3070000000048,29910,1078,27.74582560296846,5,1,0,True
|
||||||
|
tool_research_00039,tool_research,evidence_collection,4255.195999999998,21270,1412,15.063739376770538,5,1,0,True
|
||||||
|
tool_research_00040,tool_research,evidence_collection,2188.941,24341,4087,5.955713237093223,5,1,0,True
|
||||||
|
tool_research_00041,tool_research,evidence_collection,2437.484999999995,4248,1853,2.2924986508364813,3,0,0,True
|
||||||
|
tool_research_00042,tool_research,evidence_collection,2187.698999999995,8817,2395,3.681419624217119,3,0,0,True
|
||||||
|
tool_research_00043,tool_research,evidence_collection,5124.787999999995,36771,2156,17.055194805194805,9,2,0,True
|
||||||
|
tool_research_00044,tool_research,evidence_collection,2136.277000000007,23014,3951,5.824854467223488,5,1,0,True
|
||||||
|
tool_research_00045,tool_research,evidence_collection,2712.2329999999974,13715,4904,2.7966965742251224,5,1,0,True
|
||||||
|
tool_research_00046,tool_research,evidence_collection,2093.755999999999,27149,2497,10.872647176611935,3,0,0,True
|
||||||
|
tool_research_00047,tool_research,evidence_collection,3331.8359999999957,10694,2450,4.364897959183674,3,0,0,True
|
||||||
|
tool_research_00048,tool_research,evidence_collection,2496.739000000005,23797,420,56.65952380952381,5,1,0,True
|
||||||
|
tool_research_00049,tool_research,evidence_collection,3162.072999999992,37174,6914,5.376627133352618,5,1,0,True
|
||||||
|
tool_research_00050,tool_research,evidence_collection,3349.9610000000075,20453,1503,13.608117099135063,5,1,0,True
|
||||||
|
tool_research_00051,tool_research,evidence_collection,2585.735999999997,20752,4970,4.175452716297786,5,1,0,True
|
||||||
|
tool_research_00052,tool_research,evidence_collection,2241.700999999992,18226,1014,17.974358974358974,5,1,0,True
|
||||||
|
tool_research_00053,tool_research,evidence_collection,1681.621000000007,20527,1964,10.451629327902241,5,1,0,True
|
||||||
|
tool_research_00054,tool_research,evidence_collection,4616.5809999999965,14793,1020,14.50294117647059,5,1,0,True
|
||||||
|
tool_research_00055,tool_research,evidence_collection,1986.9599999999964,36674,1499,24.46564376250834,5,1,0,True
|
||||||
|
tool_research_00056,tool_research,evidence_collection,3329.991000000007,21611,2138,10.108044901777362,5,1,0,True
|
||||||
|
tool_research_00057,tool_research,evidence_collection,1935.023000000001,11508,3856,2.9844398340248963,3,0,0,True
|
||||||
|
tool_research_00058,tool_research,evidence_collection,2901.527999999999,31197,1663,18.759470835838844,5,1,0,True
|
||||||
|
tool_research_00059,tool_research,evidence_collection,3321.638000000007,11370,816,13.933823529411764,3,0,0,True
|
||||||
|
tool_research_00060,tool_research,evidence_collection,3683.4269999999947,22492,1908,11.78825995807128,5,1,0,True
|
||||||
|
tool_research_00061,tool_research,evidence_collection,2848.731999999998,18988,1760,10.788636363636364,5,1,0,True
|
||||||
|
tool_research_00062,tool_research,evidence_collection,2672.6560000000036,11461,1837,6.238976592270006,5,1,0,True
|
||||||
|
tool_research_00063,tool_research,evidence_collection,2669.6290000000004,12186,2987,4.079678607298293,3,0,0,True
|
||||||
|
tool_research_00064,tool_research,evidence_collection,1733.4749999999985,20624,2281,9.041648399824638,5,1,0,True
|
||||||
|
tool_research_00065,tool_research,evidence_collection,1969.7800000000002,37289,2064,18.066375968992247,5,1,0,True
|
||||||
|
tool_research_00066,tool_research,evidence_collection,2515.674000000004,27763,948,29.285864978902953,5,1,0,True
|
||||||
|
tool_research_00067,tool_research,evidence_collection,3096.862999999999,25812,1096,23.55109489051095,5,1,0,True
|
||||||
|
tool_research_00068,tool_research,evidence_collection,2049.244999999999,27451,1646,16.677399756986635,5,1,0,True
|
||||||
|
tool_research_00069,tool_research,evidence_collection,1731.0909999999922,25997,2954,8.800609343263371,5,1,0,True
|
||||||
|
tool_research_00070,tool_research,evidence_collection,3312.636999999995,31567,3135,10.069218500797447,5,1,0,True
|
||||||
|
tool_research_00071,tool_research,evidence_collection,1862.328000000005,20908,2780,7.520863309352518,5,1,0,True
|
||||||
|
tool_research_00072,tool_research,evidence_collection,3047.896000000009,54132,1025,52.81170731707317,5,1,0,True
|
||||||
|
tool_research_00073,tool_research,evidence_collection,2561.8630000000026,21435,4790,4.474947807933194,5,1,0,True
|
||||||
|
tool_research_00074,tool_research,evidence_collection,4560.357999999994,79773,2385,33.44779874213837,13,3,0,True
|
||||||
|
tool_research_00075,tool_research,evidence_collection,3243.395000000007,7843,2993,2.6204477113264284,3,0,0,True
|
||||||
|
tool_research_00076,tool_research,evidence_collection,2738.9289999999987,23031,2795,8.240071556350626,5,1,0,True
|
||||||
|
tool_research_00077,tool_research,evidence_collection,5143.501999999998,26921,3689,7.297641637300082,3,0,0,True
|
||||||
|
tool_research_00078,tool_research,evidence_collection,2541.331999999997,10770,5383,2.000743080066877,3,0,0,True
|
||||||
|
tool_research_00079,tool_research,evidence_collection,4596.682000000002,30560,3128,9.769820971867007,9,2,0,True
|
||||||
|
tool_research_00080,tool_research,evidence_collection,2516.4350000000013,10007,1693,5.910809214412286,3,0,0,True
|
||||||
|
tool_research_00081,tool_research,evidence_collection,3084.7260000000033,17944,830,21.619277108433735,5,1,0,True
|
||||||
|
tool_research_00082,tool_research,evidence_collection,2914.749999999998,25400,982,25.865580448065174,5,1,0,True
|
||||||
|
tool_research_00083,tool_research,evidence_collection,2230.4889999999914,24604,2144,11.475746268656716,5,1,0,True
|
||||||
|
tool_research_00084,tool_research,evidence_collection,3443.204000000009,24850,739,33.62652232746955,5,1,0,True
|
||||||
|
tool_research_00085,tool_research,evidence_collection,5764.497000000006,31697,6531,4.853314959424284,6,2,1,True
|
||||||
|
tool_research_00086,tool_research,evidence_collection,2663.731999999996,14968,967,15.478800413650465,5,1,0,True
|
||||||
|
tool_research_00087,tool_research,evidence_collection,1660.2049999999906,10680,3250,3.286153846153846,5,1,0,True
|
||||||
|
tool_research_00088,tool_research,evidence_collection,1999.1580000000085,11601,3306,3.5090744101633393,3,0,0,True
|
||||||
|
tool_research_00089,tool_research,evidence_collection,2695.4450000000065,10711,3265,3.280551301684533,5,1,0,True
|
||||||
|
tool_research_00090,tool_research,evidence_collection,5276.9389999999985,15851,1980,8.005555555555556,5,1,0,True
|
||||||
|
tool_research_00091,tool_research,evidence_collection,5764.26699999999,27062,2151,12.581125058112505,6,2,1,True
|
||||||
|
tool_research_00092,tool_research,evidence_collection,1482.0979999999936,17916,936,19.141025641025642,5,1,0,True
|
||||||
|
tool_research_00093,tool_research,evidence_collection,2160.2140000000104,36749,2052,17.908869395711502,5,1,0,True
|
||||||
|
tool_research_00094,tool_research,evidence_collection,3420.692000000017,15674,8252,1.8994183228308288,3,0,0,True
|
||||||
|
tool_research_00095,tool_research,evidence_collection,1882.05099999999,19776,2123,9.315120113047573,5,1,0,True
|
||||||
|
tool_research_00096,tool_research,evidence_collection,3586.549000000005,36710,1309,28.044308632543927,5,1,0,True
|
||||||
|
tool_research_00097,tool_research,evidence_collection,1987.364999999997,14435,3124,4.62067861715749,5,1,0,True
|
||||||
|
tool_research_00098,tool_research,evidence_collection,3917.211000000009,16232,1378,11.779390420899855,6,2,1,True
|
||||||
|
tool_research_00099,tool_research,evidence_collection,4009.145999999987,74407,2235,33.2917225950783,5,1,0,True
|
||||||
|
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
Binary file not shown.
|
After Width: | Height: | Size: 48 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 112 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 87 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 144 KiB |
@@ -0,0 +1,183 @@
|
|||||||
|
{
|
||||||
|
"global": {
|
||||||
|
"transition_probabilities": {
|
||||||
|
"CallTool->Think": 0.9891472868217054,
|
||||||
|
"Idle->Think": 0.9911167512690355,
|
||||||
|
"Retry->CallTool": 0.9176470588235294,
|
||||||
|
"Think->CallAgent": 0.12293086660175268,
|
||||||
|
"Think->CallTool": 0.15603700097370984,
|
||||||
|
"Think->Failed": 0.0017039922103213243,
|
||||||
|
"Think->Send": 0.5148490749756572,
|
||||||
|
"Think->Split": 0.20374878286270692,
|
||||||
|
"Wait->Failed": 0.36363636363636365
|
||||||
|
},
|
||||||
|
"duration_ms": {
|
||||||
|
"mean": 285.679917634447,
|
||||||
|
"std": 550.6643665581886,
|
||||||
|
"cv": 1.9275571454862042,
|
||||||
|
"p50": 8.0,
|
||||||
|
"p95": 1323.2897999999977,
|
||||||
|
"count": 5597
|
||||||
|
},
|
||||||
|
"message_size_bytes": {
|
||||||
|
"mean": 5570.409505092013,
|
||||||
|
"std": 6586.248171004335,
|
||||||
|
"cv": 1.1823633729232519,
|
||||||
|
"p50": 3158.0,
|
||||||
|
"p95": 18393.199999999997,
|
||||||
|
"count": 5597
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"task_types": {
|
||||||
|
"collaborative_analysis": {
|
||||||
|
"phase": "multi_agent_synthesis",
|
||||||
|
"task_count": 260,
|
||||||
|
"tool_probability": 0.4806201550387597,
|
||||||
|
"split_probability": 0.8192307692307692,
|
||||||
|
"mean_subtasks": 2.976923076923077,
|
||||||
|
"retry_probability": 0.12470588235294118,
|
||||||
|
"mean_retries_if_any": 1.1627906976744187,
|
||||||
|
"failure_probability": 0.007692307692307693,
|
||||||
|
"think_time_ms": {
|
||||||
|
"mean": 1012.4040078828829,
|
||||||
|
"std": 819.189229985454,
|
||||||
|
"cv": 0.8091524960460444,
|
||||||
|
"p50": 802.0150000000001,
|
||||||
|
"p95": 2609.3335999999995,
|
||||||
|
"count": 888
|
||||||
|
},
|
||||||
|
"external_size_bytes": {
|
||||||
|
"mean": 4103.692307692308,
|
||||||
|
"std": 3820.340621837784,
|
||||||
|
"cv": 0.9309520147688984,
|
||||||
|
"p50": 3039.5,
|
||||||
|
"p95": 11107.699999999995,
|
||||||
|
"count": 260
|
||||||
|
},
|
||||||
|
"request_size_bytes": {
|
||||||
|
"mean": 3936.623019182652,
|
||||||
|
"std": 3891.679972776879,
|
||||||
|
"cv": 0.9885833501997089,
|
||||||
|
"p50": 2796.0,
|
||||||
|
"p95": 10800.3,
|
||||||
|
"count": 1199
|
||||||
|
},
|
||||||
|
"response_size_bytes": {
|
||||||
|
"mean": 11829.127142857144,
|
||||||
|
"std": 8952.816088861513,
|
||||||
|
"cv": 0.7568450301312002,
|
||||||
|
"p50": 9558.5,
|
||||||
|
"p95": 29122.249999999993,
|
||||||
|
"count": 1400
|
||||||
|
},
|
||||||
|
"message_count_per_task": {
|
||||||
|
"mean": 11.01923076923077,
|
||||||
|
"std": 5.376910696585285,
|
||||||
|
"cv": 0.48795699166218987,
|
||||||
|
"p50": 10.0,
|
||||||
|
"p95": 21.049999999999983,
|
||||||
|
"count": 260
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"simple_qa": {
|
||||||
|
"phase": "answering",
|
||||||
|
"task_count": 260,
|
||||||
|
"tool_probability": 0.13740458015267176,
|
||||||
|
"split_probability": 0.03461538461538462,
|
||||||
|
"mean_subtasks": 1.0076923076923077,
|
||||||
|
"retry_probability": 0.02702702702702703,
|
||||||
|
"mean_retries_if_any": 1.0,
|
||||||
|
"failure_probability": 0.0,
|
||||||
|
"think_time_ms": {
|
||||||
|
"mean": 315.74304316546767,
|
||||||
|
"std": 193.59344702605458,
|
||||||
|
"cv": 0.6131360649634342,
|
||||||
|
"p50": 262.062,
|
||||||
|
"p95": 693.3605,
|
||||||
|
"count": 556
|
||||||
|
},
|
||||||
|
"external_size_bytes": {
|
||||||
|
"mean": 1285.7846153846153,
|
||||||
|
"std": 562.841233039538,
|
||||||
|
"cv": 0.43774145864327046,
|
||||||
|
"p50": 1167.5,
|
||||||
|
"p95": 2332.1,
|
||||||
|
"count": 260
|
||||||
|
},
|
||||||
|
"request_size_bytes": {
|
||||||
|
"mean": 1646.314381270903,
|
||||||
|
"std": 819.0778288985601,
|
||||||
|
"cv": 0.4975221247027301,
|
||||||
|
"p50": 1537.0,
|
||||||
|
"p95": 3065.4999999999995,
|
||||||
|
"count": 299
|
||||||
|
},
|
||||||
|
"response_size_bytes": {
|
||||||
|
"mean": 1909.388888888889,
|
||||||
|
"std": 1208.6473367395993,
|
||||||
|
"cv": 0.6330021839830309,
|
||||||
|
"p50": 1577.5,
|
||||||
|
"p95": 4346.099999999999,
|
||||||
|
"count": 558
|
||||||
|
},
|
||||||
|
"message_count_per_task": {
|
||||||
|
"mean": 4.296153846153846,
|
||||||
|
"std": 0.7184281823915581,
|
||||||
|
"cv": 0.16722589742328123,
|
||||||
|
"p50": 4.0,
|
||||||
|
"p95": 6.0,
|
||||||
|
"count": 260
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"tool_research": {
|
||||||
|
"phase": "evidence_collection",
|
||||||
|
"task_count": 260,
|
||||||
|
"tool_probability": 0.7631578947368421,
|
||||||
|
"split_probability": 0.2076923076923077,
|
||||||
|
"mean_subtasks": 1.1692307692307693,
|
||||||
|
"retry_probability": 0.09019607843137255,
|
||||||
|
"mean_retries_if_any": 1.0952380952380953,
|
||||||
|
"failure_probability": 0.0038461538461538464,
|
||||||
|
"think_time_ms": {
|
||||||
|
"mean": 660.7411426666666,
|
||||||
|
"std": 548.0475005635645,
|
||||||
|
"cv": 0.8294435826286146,
|
||||||
|
"p50": 513.7415,
|
||||||
|
"p95": 1615.4994999999997,
|
||||||
|
"count": 750
|
||||||
|
},
|
||||||
|
"external_size_bytes": {
|
||||||
|
"mean": 2591.223076923077,
|
||||||
|
"std": 1938.9205968801086,
|
||||||
|
"cv": 0.7482646377101818,
|
||||||
|
"p50": 2023.0,
|
||||||
|
"p95": 6498.299999999999,
|
||||||
|
"count": 260
|
||||||
|
},
|
||||||
|
"request_size_bytes": {
|
||||||
|
"mean": 2178.8425760286227,
|
||||||
|
"std": 1869.411620553448,
|
||||||
|
"cv": 0.8579837942954215,
|
||||||
|
"p50": 1586.0,
|
||||||
|
"p95": 6085.500000000001,
|
||||||
|
"count": 559
|
||||||
|
},
|
||||||
|
"response_size_bytes": {
|
||||||
|
"mean": 6327.029003783102,
|
||||||
|
"std": 4973.701881543845,
|
||||||
|
"cv": 0.7861038535732859,
|
||||||
|
"p50": 4987.0,
|
||||||
|
"p95": 15569.8,
|
||||||
|
"count": 793
|
||||||
|
},
|
||||||
|
"message_count_per_task": {
|
||||||
|
"mean": 6.211538461538462,
|
||||||
|
"std": 2.097035805988451,
|
||||||
|
"cv": 0.33760328765139147,
|
||||||
|
"p50": 6.0,
|
||||||
|
"p95": 11.0,
|
||||||
|
"count": 260
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
timeout_probability,max_retries,success_rate,mean_retries,mean_amplification,mean_latency_ms
|
||||||
|
0.0,0,1.0,0.0,9.791625846103754,1611.6456398964472
|
||||||
|
0.0,1,1.0,0.0,9.929872318854873,1810.7223041670222
|
||||||
|
0.0,2,1.0,0.0,8.566505830883267,1646.5055480731012
|
||||||
|
0.0,3,1.0,0.0,9.466712608627542,1652.6593639522055
|
||||||
|
0.0,5,1.0,0.0,9.662682945914268,1756.6494872943697
|
||||||
|
0.03,0,0.9625,0.0,9.236763247092442,1764.2494023015984
|
||||||
|
0.03,1,1.0,0.016666666666666666,9.49838873538483,1668.745910772185
|
||||||
|
0.03,2,1.0,0.0375,9.40180231370975,1753.199216159701
|
||||||
|
0.03,3,1.0,0.016666666666666666,10.022457643868133,1860.8835997078063
|
||||||
|
0.03,5,1.0,0.025,9.019935163210244,1900.697800739017
|
||||||
|
0.08,0,0.9291666666666667,0.0,9.92154310687105,1719.1310133789257
|
||||||
|
0.08,1,0.9958333333333333,0.05416666666666667,9.48779578533714,1843.49537077021
|
||||||
|
0.08,2,0.9958333333333333,0.07083333333333333,9.542258282931957,1902.6419815786642
|
||||||
|
0.08,3,1.0,0.05,9.887456874772154,1817.7727139655656
|
||||||
|
0.08,5,1.0,0.09166666666666666,9.097284092135592,1904.8547916418988
|
||||||
|
0.15,0,0.8541666666666666,0.0,8.835720371687312,1571.6219931658118
|
||||||
|
0.15,1,0.9791666666666666,0.09583333333333334,9.122618265595515,1955.8897478710594
|
||||||
|
0.15,2,1.0,0.10833333333333334,9.846728400807162,1980.994814000937
|
||||||
|
0.15,3,1.0,0.12083333333333333,9.51555007863001,1950.3948320063673
|
||||||
|
0.15,5,1.0,0.19166666666666668,10.257406318574677,2173.315967936333
|
||||||
|
0.25,0,0.8333333333333334,0.0,8.62914384683803,1654.824555364319
|
||||||
|
0.25,1,0.9541666666666667,0.16666666666666666,8.984142037708278,2090.160252075161
|
||||||
|
0.25,2,0.9875,0.2625,9.642823436259429,2393.4294606209505
|
||||||
|
0.25,3,1.0,0.22916666666666666,8.7946687060034,2226.263869969628
|
||||||
|
0.25,5,1.0,0.19583333333333333,9.066500470500229,2110.276874421043
|
||||||
|
@@ -0,0 +1,9 @@
|
|||||||
|
arrival_rate,offered_tasks,completed_tasks,throughput_per_second,mean_delay_ms,p95_delay_ms,queue_peak,drop_rate,success_rate,mean_amplification
|
||||||
|
0.5,98,85,0.53125,5789.822235500761,12757.162733844745,5,0.0,1.0,31.470196724444556
|
||||||
|
1.0,176,160,1.0,31384.496417526076,42781.94943988779,34,0.0,1.0,31.986629448479086
|
||||||
|
2.0,352,317,1.98125,153100.48173718844,266221.0600580607,211,0.0,0.9968454258675079,27.062374263992844
|
||||||
|
4.0,687,574,3.5875,371545.11383159773,654462.4416805771,500,0.06841339155749636,1.0,32.343350031249344
|
||||||
|
6.0,1095,518,3.2375,403282.1583010225,624405.7892324593,500,0.410958904109589,0.9980694980694981,28.847411906142742
|
||||||
|
8.0,1433,477,2.98125,453122.6068650545,634995.5685232729,500,0.5422191207257502,1.0,28.88759065286518
|
||||||
|
10.0,1829,465,2.90625,460685.0783967892,629246.0471832517,500,0.644614543466375,0.9978494623655914,29.343167280152798
|
||||||
|
12.0,2201,410,2.5625,502425.69206713024,633541.2012092929,500,0.7019536574284416,0.9975609756097561,31.051514679264738
|
||||||
|
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"best_model_by_internal_bytes_mape": {
|
||||||
|
"model": "full_model",
|
||||||
|
"metric": "internal_bytes",
|
||||||
|
"mae": 8514.790993271408,
|
||||||
|
"rmse": 10202.035682525451,
|
||||||
|
"mape_percent": 25.132613550059535
|
||||||
|
},
|
||||||
|
"best_model_by_latency_mape": {
|
||||||
|
"model": "full_model",
|
||||||
|
"metric": "latency_ms",
|
||||||
|
"mae": 1740.55513184213,
|
||||||
|
"rmse": 2061.3161939037764,
|
||||||
|
"mape_percent": 42.13888590021146
|
||||||
|
},
|
||||||
|
"max_stress_arrival_rate": 12.0,
|
||||||
|
"max_observed_p95_delay_ms": 654462.4416805771
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
model,metric,mae,rmse,mape_percent
|
||||||
|
full_model,internal_bytes,8514.790993271408,10202.035682525451,25.132613550059535
|
||||||
|
full_model,latency_ms,1740.55513184213,2061.3161939037764,42.13888590021146
|
||||||
|
full_model,message_count,0.24833333333333366,0.34639011404933506,4.4460827599229225
|
||||||
|
full_model,tool_calls,0.0977777777777778,0.12750453877179202,8.967944987269812
|
||||||
|
full_model,amplification,2.1844309474987855,3.1463454795356744,17.403062466244346
|
||||||
|
no_context,internal_bytes,28106.101475356598,38300.17877551533,107.12005537801083
|
||||||
|
no_context,latency_ms,3059.4684226124723,4030.5538811505185,68.83144127074493
|
||||||
|
no_context,message_count,2.332777777777778,3.1734000051352798,34.22052281701404
|
||||||
|
no_context,tool_calls,0.645,0.7287946670130528,188.58078578942232
|
||||||
|
no_context,amplification,10.861414695027626,13.961977843833159,67.33834604907454
|
||||||
|
static_mean,internal_bytes,29834.93333333333,32709.783175317236,230.39064998346973
|
||||||
|
static_mean,latency_ms,2757.6886622222223,3058.6901465213614,119.58343818448623
|
||||||
|
static_mean,message_count,2.5155555555555558,2.7855500155066126,48.03496553983882
|
||||||
|
static_mean,tool_calls,0.5444444444444444,0.6355225321645873,264.95059144468667
|
||||||
|
static_mean,amplification,10.71876769134868,12.026846682885607,111.10365289444462
|
||||||
|
@@ -0,0 +1,10 @@
|
|||||||
|
task_type,model,actual_internal_bytes,predicted_internal_bytes,actual_latency_ms,predicted_latency_ms,actual_message_count,predicted_message_count,actual_tool_calls,predicted_tool_calls,actual_amplification,predicted_amplification
|
||||||
|
collaborative_analysis,static_mean,82782.45,38030.049999999996,8428.771760000001,4292.238766666668,9.88,6.1066666666666665,1.66,0.9266666666666667,33.798140646040636,17.719989109017618
|
||||||
|
collaborative_analysis,no_context,82782.45,17953.655088125477,8428.771760000001,1674.760201302985,9.88,4.575,1.66,0.55,33.798140646040636,10.538207426716697
|
||||||
|
collaborative_analysis,full_model,82782.45,68099.5326730428,8428.771760000001,5276.731101216728,9.88,9.741666666666667,1.66,1.75,33.798140646040636,33.290090674565
|
||||||
|
simple_qa,static_mean,5514.23,38030.049999999996,1128.43623,4292.238766666668,3.24,6.1066666666666665,0.11,0.9266666666666667,4.875984197964249,17.719989109017618
|
||||||
|
simple_qa,no_context,5514.23,17261.226304896398,1128.43623,2040.1508029795593,3.24,4.65,0.11,0.625,4.875984197964249,9.934747754184363
|
||||||
|
simple_qa,full_model,5514.23,4423.486362645099,1128.43623,672.4328817972068,3.24,3.216666666666667,0.11,0.10833333333333334,4.875984197964249,4.216398328357828
|
||||||
|
tool_research,static_mean,25793.47,38030.049999999996,3319.5083100000024,4292.238766666668,5.2,6.1066666666666665,1.01,0.9266666666666667,14.485842483047964,17.719989109017618
|
||||||
|
tool_research,no_context,25793.47,18050.956790701126,3319.5083100000024,1806.829173839162,5.2,4.916666666666667,1.01,0.7,14.485842483047964,10.22029517350914
|
||||||
|
tool_research,full_model,25793.47,16022.75798449787,3319.5083100000024,1705.8869214596782,5.2,4.616666666666666,1.01,0.8083333333333333,14.485842483047964,9.100185481633664
|
||||||
|
@@ -0,0 +1,37 @@
|
|||||||
|
seed: 20260813
|
||||||
|
|
||||||
|
generation:
|
||||||
|
train_tasks_per_type: 260
|
||||||
|
test_tasks_per_type: 100
|
||||||
|
task_types:
|
||||||
|
- simple_qa
|
||||||
|
- tool_research
|
||||||
|
- collaborative_analysis
|
||||||
|
llm:
|
||||||
|
temperature: 0.5
|
||||||
|
timeout_seconds: 90
|
||||||
|
max_tokens: 3500
|
||||||
|
|
||||||
|
network:
|
||||||
|
agent_count: 8
|
||||||
|
tool_count: 3
|
||||||
|
coordinator: agent_0
|
||||||
|
default_link_bandwidth_mbps: 20
|
||||||
|
default_link_delay_ms: 8
|
||||||
|
max_concurrency: 4
|
||||||
|
queue_capacity: 500
|
||||||
|
|
||||||
|
simulation:
|
||||||
|
replications: 16
|
||||||
|
timeout_ms: 2200
|
||||||
|
max_retries: 2
|
||||||
|
retry_backoff_ms: 180
|
||||||
|
|
||||||
|
experiments:
|
||||||
|
validation_tasks_per_scenario: 120
|
||||||
|
arrival_rates_per_second: [0.5, 1, 2, 4, 6, 8, 10, 12]
|
||||||
|
stress_duration_seconds: 180
|
||||||
|
stress_warmup_seconds: 20
|
||||||
|
timeout_probabilities: [0.0, 0.03, 0.08, 0.15, 0.25]
|
||||||
|
retry_limits: [0, 1, 2, 3, 5]
|
||||||
|
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"generator": "local_builtin"
|
||||||
|
},
|
||||||
|
"profiles": [
|
||||||
|
{
|
||||||
|
"task_type": "simple_qa",
|
||||||
|
"description": "单节点即可完成的简短问答，少量情况下调用工具。",
|
||||||
|
"phase": "answering",
|
||||||
|
"tool_probability": 0.1,
|
||||||
|
"split_probability": 0.05,
|
||||||
|
"mean_subtasks": 1.2,
|
||||||
|
"timeout_probability": 0.02,
|
||||||
|
"failure_probability": 0.01,
|
||||||
|
"think_time_ms_mean": 420.0,
|
||||||
|
"think_time_cv": 0.55,
|
||||||
|
"request_size_bytes_mean": 1800.0,
|
||||||
|
"response_size_bytes_mean": 2600.0,
|
||||||
|
"message_size_cv": 0.45
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"task_type": "tool_research",
|
||||||
|
"description": "需要搜索、数据库或工具结果的研究任务。",
|
||||||
|
"phase": "evidence_collection",
|
||||||
|
"tool_probability": 0.78,
|
||||||
|
"split_probability": 0.2,
|
||||||
|
"mean_subtasks": 1.8,
|
||||||
|
"timeout_probability": 0.07,
|
||||||
|
"failure_probability": 0.025,
|
||||||
|
"think_time_ms_mean": 900.0,
|
||||||
|
"think_time_cv": 0.75,
|
||||||
|
"request_size_bytes_mean": 3200.0,
|
||||||
|
"response_size_bytes_mean": 8500.0,
|
||||||
|
"message_size_cv": 0.7
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"task_type": "collaborative_analysis",
|
||||||
|
"description": "协调多个执行 Agent 并汇总结果的复杂分析任务。",
|
||||||
|
"phase": "multi_agent_synthesis",
|
||||||
|
"tool_probability": 0.48,
|
||||||
|
"split_probability": 0.82,
|
||||||
|
"mean_subtasks": 3.4,
|
||||||
|
"timeout_probability": 0.09,
|
||||||
|
"failure_probability": 0.035,
|
||||||
|
"think_time_ms_mean": 1450.0,
|
||||||
|
"think_time_cv": 0.85,
|
||||||
|
"request_size_bytes_mean": 5200.0,
|
||||||
|
"response_size_bytes_mean": 11800.0,
|
||||||
|
"message_size_cv": 0.8
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,301 @@
|
|||||||
|
task_id,task_type,task_phase,latency_ms,internal_bytes,input_bytes,amplification,message_count,tool_calls,retries,success
|
||||||
|
collaborative_analysis_00000,collaborative_analysis,multi_agent_synthesis,6111.839000000004,62448,1205,51.824066390041494,7,1,0,True
|
||||||
|
collaborative_analysis_00001,collaborative_analysis,multi_agent_synthesis,7858.875999999981,57236,2452,23.34257748776509,7,0,0,True
|
||||||
|
collaborative_analysis_00002,collaborative_analysis,multi_agent_synthesis,3745.010999999977,36184,6916,5.231925968768074,5,1,0,True
|
||||||
|
collaborative_analysis_00003,collaborative_analysis,multi_agent_synthesis,3370.6700000000183,15600,1146,13.612565445026178,3,0,0,True
|
||||||
|
collaborative_analysis_00004,collaborative_analysis,multi_agent_synthesis,3970.753000000002,26927,5695,4.728182616330114,5,1,0,True
|
||||||
|
collaborative_analysis_00005,collaborative_analysis,multi_agent_synthesis,3692.181000000005,9783,2845,3.438664323374341,3,0,0,True
|
||||||
|
collaborative_analysis_00006,collaborative_analysis,multi_agent_synthesis,12027.614,101344,5088,19.91823899371069,9,1,0,True
|
||||||
|
collaborative_analysis_00007,collaborative_analysis,multi_agent_synthesis,3174.7689999999975,13427,10177,1.3193475483934363,3,0,0,True
|
||||||
|
collaborative_analysis_00008,collaborative_analysis,multi_agent_synthesis,4553.435000000008,106516,5924,17.98041863605672,13,2,0,True
|
||||||
|
collaborative_analysis_00009,collaborative_analysis,multi_agent_synthesis,20393.187000000013,136393,8870,15.376888387824126,19,3,0,True
|
||||||
|
collaborative_analysis_00010,collaborative_analysis,multi_agent_synthesis,9716.779000000002,43830,1420,30.866197183098592,9,2,0,True
|
||||||
|
collaborative_analysis_00011,collaborative_analysis,multi_agent_synthesis,13658.126999999979,113877,6803,16.73923269145965,13,2,0,True
|
||||||
|
collaborative_analysis_00012,collaborative_analysis,multi_agent_synthesis,5092.640999999986,138420,4391,31.523570940560237,13,2,0,True
|
||||||
|
collaborative_analysis_00013,collaborative_analysis,multi_agent_synthesis,2393.601999999987,13198,8494,1.5538026842477042,3,0,0,True
|
||||||
|
collaborative_analysis_00014,collaborative_analysis,multi_agent_synthesis,7330.83400000001,65750,876,75.05707762557077,11,2,0,True
|
||||||
|
collaborative_analysis_00015,collaborative_analysis,multi_agent_synthesis,16492.502,180400,2157,83.63467779323133,23,3,0,True
|
||||||
|
collaborative_analysis_00016,collaborative_analysis,multi_agent_synthesis,16875.831000000006,205302,2605,78.81074856046065,19,3,0,True
|
||||||
|
collaborative_analysis_00017,collaborative_analysis,multi_agent_synthesis,9850.209000000006,147782,1988,74.33702213279678,19,3,0,True
|
||||||
|
collaborative_analysis_00018,collaborative_analysis,multi_agent_synthesis,10007.738999999987,126126,4218,29.90184921763869,15,2,0,True
|
||||||
|
collaborative_analysis_00019,collaborative_analysis,multi_agent_synthesis,11525.198999999986,106483,1192,89.33137583892618,7,1,0,True
|
||||||
|
collaborative_analysis_00020,collaborative_analysis,multi_agent_synthesis,5159.906000000006,59557,2210,26.94886877828054,9,2,0,True
|
||||||
|
collaborative_analysis_00021,collaborative_analysis,multi_agent_synthesis,4791.642999999994,57467,18445,3.1155868799132556,5,0,0,True
|
||||||
|
collaborative_analysis_00022,collaborative_analysis,multi_agent_synthesis,11351.583000000006,75128,3315,22.663046757164405,9,1,0,True
|
||||||
|
collaborative_analysis_00023,collaborative_analysis,multi_agent_synthesis,8277.253999999999,39603,5217,7.591144335825187,5,1,0,True
|
||||||
|
collaborative_analysis_00024,collaborative_analysis,multi_agent_synthesis,19919.341000000004,167247,1166,143.43653516295026,23,5,0,True
|
||||||
|
collaborative_analysis_00025,collaborative_analysis,multi_agent_synthesis,9099.73500000001,69536,6843,10.161625018266841,11,1,0,True
|
||||||
|
collaborative_analysis_00026,collaborative_analysis,multi_agent_synthesis,3098.5450000000014,55189,7507,7.351671773011856,11,2,0,True
|
||||||
|
collaborative_analysis_00027,collaborative_analysis,multi_agent_synthesis,6933.150000000012,128239,5952,21.545530913978496,13,3,0,True
|
||||||
|
collaborative_analysis_00028,collaborative_analysis,multi_agent_synthesis,14684.41899999999,164414,1446,113.70262793914246,18,5,1,True
|
||||||
|
collaborative_analysis_00029,collaborative_analysis,multi_agent_synthesis,22606.593000000004,130445,2700,48.31296296296296,13,3,2,True
|
||||||
|
collaborative_analysis_00030,collaborative_analysis,multi_agent_synthesis,7566.215999999997,53530,2840,18.848591549295776,7,0,0,True
|
||||||
|
collaborative_analysis_00031,collaborative_analysis,multi_agent_synthesis,9549.07,182717,560,326.28035714285716,11,2,0,True
|
||||||
|
collaborative_analysis_00032,collaborative_analysis,multi_agent_synthesis,16413.354,140512,4336,32.40590405904059,13,1,0,True
|
||||||
|
collaborative_analysis_00033,collaborative_analysis,multi_agent_synthesis,3107.8390000000127,25062,1526,16.42332896461337,3,0,0,True
|
||||||
|
collaborative_analysis_00034,collaborative_analysis,multi_agent_synthesis,2382.689999999997,37813,4410,8.57437641723356,5,1,0,True
|
||||||
|
collaborative_analysis_00035,collaborative_analysis,multi_agent_synthesis,6573.08900000001,42752,1231,34.72948822095857,9,2,0,True
|
||||||
|
collaborative_analysis_00036,collaborative_analysis,multi_agent_synthesis,14503.11400000001,179234,7724,23.20481615743138,23,6,1,True
|
||||||
|
collaborative_analysis_00037,collaborative_analysis,multi_agent_synthesis,6998.415999999992,40737,4407,9.243703199455412,5,1,0,True
|
||||||
|
collaborative_analysis_00038,collaborative_analysis,multi_agent_synthesis,9218.387000000006,76646,5615,13.650222617987533,12,3,1,True
|
||||||
|
collaborative_analysis_00039,collaborative_analysis,multi_agent_synthesis,14171.648999999974,82798,3958,20.919151086407275,13,2,0,True
|
||||||
|
collaborative_analysis_00040,collaborative_analysis,multi_agent_synthesis,16203.926999999992,192384,4272,45.03370786516854,19,2,0,True
|
||||||
|
collaborative_analysis_00041,collaborative_analysis,multi_agent_synthesis,8077.832999999999,86864,14703,5.907909950350269,9,1,0,True
|
||||||
|
collaborative_analysis_00042,collaborative_analysis,multi_agent_synthesis,11980.147000000017,175669,4129,42.545168321627514,17,2,0,True
|
||||||
|
collaborative_analysis_00043,collaborative_analysis,multi_agent_synthesis,8383.41299999999,86553,4556,18.99758560140474,11,1,0,True
|
||||||
|
collaborative_analysis_00044,collaborative_analysis,multi_agent_synthesis,3781.3569999999854,21537,2248,9.580516014234876,5,1,0,True
|
||||||
|
collaborative_analysis_00045,collaborative_analysis,multi_agent_synthesis,2871.480999999989,29178,3997,7.299974981235927,5,1,0,True
|
||||||
|
collaborative_analysis_00046,collaborative_analysis,multi_agent_synthesis,1987.737999999979,21361,8073,2.6459804285891244,3,0,0,True
|
||||||
|
collaborative_analysis_00047,collaborative_analysis,multi_agent_synthesis,10075.866999999987,219312,9941,22.061362036012472,17,3,0,True
|
||||||
|
collaborative_analysis_00048,collaborative_analysis,multi_agent_synthesis,6179.957999999999,70354,2470,28.4834008097166,5,1,0,True
|
||||||
|
collaborative_analysis_00049,collaborative_analysis,multi_agent_synthesis,3145.8080000000164,28609,4669,6.127436281859071,3,0,0,True
|
||||||
|
collaborative_analysis_00050,collaborative_analysis,multi_agent_synthesis,6249.4649999999865,28317,882,32.105442176870746,7,1,0,True
|
||||||
|
collaborative_analysis_00051,collaborative_analysis,multi_agent_synthesis,11702.425000000005,187198,1275,146.82196078431372,17,2,0,True
|
||||||
|
collaborative_analysis_00052,collaborative_analysis,multi_agent_synthesis,10426.145999999988,122179,1403,87.08410548823949,13,2,0,True
|
||||||
|
collaborative_analysis_00053,collaborative_analysis,multi_agent_synthesis,3776.5890000000013,10701,3073,3.482264887731858,3,0,0,True
|
||||||
|
collaborative_analysis_00054,collaborative_analysis,multi_agent_synthesis,6411.059999999992,122716,3644,33.6761800219539,13,3,0,True
|
||||||
|
collaborative_analysis_00055,collaborative_analysis,multi_agent_synthesis,9190.782999999981,87827,8202,10.707998049256279,13,1,0,True
|
||||||
|
collaborative_analysis_00056,collaborative_analysis,multi_agent_synthesis,26072.79299999999,196393,2732,71.88616398243046,22,5,1,True
|
||||||
|
collaborative_analysis_00057,collaborative_analysis,multi_agent_synthesis,6656.256000000013,33081,7347,4.502654144548796,7,1,0,True
|
||||||
|
collaborative_analysis_00058,collaborative_analysis,multi_agent_synthesis,5274.61199999999,58229,9303,6.259163710630979,8,2,1,True
|
||||||
|
collaborative_analysis_00059,collaborative_analysis,multi_agent_synthesis,5612.335999999999,17516,7543,2.3221529895267135,3,0,0,True
|
||||||
|
collaborative_analysis_00060,collaborative_analysis,multi_agent_synthesis,7323.184999999995,88159,6584,13.389884568651276,5,1,0,True
|
||||||
|
collaborative_analysis_00061,collaborative_analysis,multi_agent_synthesis,13041.410999999982,39582,4651,8.510427864975274,5,0,0,True
|
||||||
|
collaborative_analysis_00062,collaborative_analysis,multi_agent_synthesis,5737.938000000014,55676,4041,13.777777777777779,11,2,0,True
|
||||||
|
collaborative_analysis_00063,collaborative_analysis,multi_agent_synthesis,13871.848999999998,111676,5758,19.39492879472039,18,5,1,True
|
||||||
|
collaborative_analysis_00064,collaborative_analysis,multi_agent_synthesis,14159.884000000005,93620,743,126.00269179004037,11,2,0,True
|
||||||
|
collaborative_analysis_00065,collaborative_analysis,multi_agent_synthesis,2143.2680000000064,44223,2414,18.319386909693456,5,0,0,True
|
||||||
|
collaborative_analysis_00066,collaborative_analysis,multi_agent_synthesis,9103.234999999984,120161,2301,52.22120817036071,11,0,0,True
|
||||||
|
collaborative_analysis_00067,collaborative_analysis,multi_agent_synthesis,7975.786999999997,51818,2211,23.43645409317051,9,2,0,True
|
||||||
|
collaborative_analysis_00068,collaborative_analysis,multi_agent_synthesis,10306.556999999997,143550,1168,122.90239726027397,15,3,0,True
|
||||||
|
collaborative_analysis_00069,collaborative_analysis,multi_agent_synthesis,10491.949999999975,88142,1034,85.24371373307544,13,3,0,True
|
||||||
|
collaborative_analysis_00070,collaborative_analysis,multi_agent_synthesis,5671.043999999994,92836,4373,21.229361994054425,7,1,0,True
|
||||||
|
collaborative_analysis_00071,collaborative_analysis,multi_agent_synthesis,2233.918000000017,35792,908,39.418502202643175,5,1,0,True
|
||||||
|
collaborative_analysis_00072,collaborative_analysis,multi_agent_synthesis,2779.2799999999997,19132,1140,16.782456140350877,3,0,0,True
|
||||||
|
collaborative_analysis_00073,collaborative_analysis,multi_agent_synthesis,5933.890999999989,53137,2409,22.057700290577003,7,1,0,True
|
||||||
|
collaborative_analysis_00074,collaborative_analysis,multi_agent_synthesis,11494.603000000012,198018,8671,22.836812363049244,15,5,1,True
|
||||||
|
collaborative_analysis_00075,collaborative_analysis,multi_agent_synthesis,3800.8010000000068,92249,1891,48.78318350079323,7,0,0,True
|
||||||
|
collaborative_analysis_00076,collaborative_analysis,multi_agent_synthesis,15271.408000000009,110064,4836,22.759305210918114,13,4,2,True
|
||||||
|
collaborative_analysis_00077,collaborative_analysis,multi_agent_synthesis,3524.1379999999936,49633,2359,21.03984739296312,7,0,0,True
|
||||||
|
collaborative_analysis_00078,collaborative_analysis,multi_agent_synthesis,1085.5840000000114,34459,1481,23.267386900742743,3,0,0,True
|
||||||
|
collaborative_analysis_00079,collaborative_analysis,multi_agent_synthesis,3396.9510000000014,7311,4899,1.492345376607471,3,0,0,True
|
||||||
|
collaborative_analysis_00080,collaborative_analysis,multi_agent_synthesis,7375.2580000000025,40747,2612,15.599923430321592,10,3,1,True
|
||||||
|
collaborative_analysis_00081,collaborative_analysis,multi_agent_synthesis,11392.599000000018,125175,5129,24.405342171963344,17,5,1,True
|
||||||
|
collaborative_analysis_00082,collaborative_analysis,multi_agent_synthesis,4221.8009999999995,46903,1193,39.3151718357083,3,0,0,True
|
||||||
|
collaborative_analysis_00083,collaborative_analysis,multi_agent_synthesis,4469.9559999999965,34392,7835,4.389534141671985,5,0,0,True
|
||||||
|
collaborative_analysis_00084,collaborative_analysis,multi_agent_synthesis,14913.923000000012,148982,4218,35.32053105737316,17,5,2,False
|
||||||
|
collaborative_analysis_00085,collaborative_analysis,multi_agent_synthesis,3174.0640000000158,26574,2863,9.281872162067762,3,0,0,True
|
||||||
|
collaborative_analysis_00086,collaborative_analysis,multi_agent_synthesis,6104.687000000013,29578,2252,13.134103019538188,6,2,1,True
|
||||||
|
collaborative_analysis_00087,collaborative_analysis,multi_agent_synthesis,3079.0619999999935,23392,7001,3.3412369661476933,3,0,0,True
|
||||||
|
collaborative_analysis_00088,collaborative_analysis,multi_agent_synthesis,15710.582000000017,123812,1087,113.90248390064397,17,3,0,True
|
||||||
|
collaborative_analysis_00089,collaborative_analysis,multi_agent_synthesis,9594.03499999999,123841,7467,16.585107807687155,15,3,0,True
|
||||||
|
collaborative_analysis_00090,collaborative_analysis,multi_agent_synthesis,8018.177000000009,35223,2432,14.483141447368421,7,1,0,True
|
||||||
|
collaborative_analysis_00091,collaborative_analysis,multi_agent_synthesis,8070.577999999983,85639,1800,47.577222222222225,9,1,0,True
|
||||||
|
collaborative_analysis_00092,collaborative_analysis,multi_agent_synthesis,6954.7670000000035,30543,2606,11.720260936300845,8,2,1,True
|
||||||
|
collaborative_analysis_00093,collaborative_analysis,multi_agent_synthesis,2334.4150000000072,9761,1259,7.75297855440826,3,0,0,True
|
||||||
|
collaborative_analysis_00094,collaborative_analysis,multi_agent_synthesis,3988.2240000000024,53918,4052,13.306515301085884,7,0,0,True
|
||||||
|
collaborative_analysis_00095,collaborative_analysis,multi_agent_synthesis,6538.421999999997,67526,4362,15.480513525905549,7,1,0,True
|
||||||
|
collaborative_analysis_00096,collaborative_analysis,multi_agent_synthesis,7050.077000000016,89764,5545,16.188277727682596,9,0,0,True
|
||||||
|
collaborative_analysis_00097,collaborative_analysis,multi_agent_synthesis,14613.135,138234,1905,72.56377952755905,18,4,1,True
|
||||||
|
collaborative_analysis_00098,collaborative_analysis,multi_agent_synthesis,5859.162999999995,34186,15338,2.2288433954883295,6,2,1,True
|
||||||
|
collaborative_analysis_00099,collaborative_analysis,multi_agent_synthesis,17759.783999999996,169187,11129,15.202354209722348,21,5,0,True
|
||||||
|
simple_qa_00000,simple_qa,answering,855.1170000000001,3448,2896,1.1906077348066297,3,0,0,True
|
||||||
|
simple_qa_00001,simple_qa,answering,1403.896,4962,1531,3.24101894186806,3,0,0,True
|
||||||
|
simple_qa_00002,simple_qa,answering,1171.8220000000001,4350,787,5.527318932655654,3,0,0,True
|
||||||
|
simple_qa_00003,simple_qa,answering,1504.4239999999998,2494,375,6.650666666666667,3,0,0,True
|
||||||
|
simple_qa_00004,simple_qa,answering,1963.5280000000002,5963,591,10.089678510998308,5,1,0,True
|
||||||
|
simple_qa_00005,simple_qa,answering,1452.445,5943,2101,2.8286530223703,3,0,0,True
|
||||||
|
simple_qa_00006,simple_qa,answering,825.0729999999998,3757,1209,3.10752688172043,3,0,0,True
|
||||||
|
simple_qa_00007,simple_qa,answering,926.895,3668,2132,1.7204502814258913,3,0,0,True
|
||||||
|
simple_qa_00008,simple_qa,answering,1005.903,4694,1529,3.0699803793328972,3,0,0,True
|
||||||
|
simple_qa_00009,simple_qa,answering,1038.875,5016,1632,3.073529411764706,3,0,0,True
|
||||||
|
simple_qa_00010,simple_qa,answering,659.4789999999992,9518,859,11.080325960419092,3,0,0,True
|
||||||
|
simple_qa_00011,simple_qa,answering,1125.273,3965,1038,3.8198458574181116,3,0,0,True
|
||||||
|
simple_qa_00012,simple_qa,answering,1077.1380000000015,5386,1219,4.418375717801476,3,0,0,True
|
||||||
|
simple_qa_00013,simple_qa,answering,1004.5659999999988,5660,889,6.366704161979753,3,0,0,True
|
||||||
|
simple_qa_00014,simple_qa,answering,996.1830000000002,4420,1161,3.8070628768303187,3,0,0,True
|
||||||
|
simple_qa_00015,simple_qa,answering,860.6720000000009,4252,1484,2.8652291105121295,3,0,0,True
|
||||||
|
simple_qa_00016,simple_qa,answering,781.1749999999993,6704,1189,5.638351555929352,3,0,0,True
|
||||||
|
simple_qa_00017,simple_qa,answering,700.6979999999992,4185,1526,2.7424639580602883,3,0,0,True
|
||||||
|
simple_qa_00018,simple_qa,answering,1115.2699999999988,5439,1075,5.05953488372093,3,0,0,True
|
||||||
|
simple_qa_00019,simple_qa,answering,850.3690000000006,3539,1636,2.16320293398533,3,0,0,True
|
||||||
|
simple_qa_00020,simple_qa,answering,823.193999999999,5658,1063,5.322671683913453,3,0,0,True
|
||||||
|
simple_qa_00021,simple_qa,answering,930.963000000002,6726,2009,3.3479342956694875,3,0,0,True
|
||||||
|
simple_qa_00022,simple_qa,answering,742.083000000001,5048,1822,2.770581778265642,3,0,0,True
|
||||||
|
simple_qa_00023,simple_qa,answering,1172.7469999999976,3674,1567,2.3446075303126994,3,0,0,True
|
||||||
|
simple_qa_00024,simple_qa,answering,604.9790000000002,7257,1006,7.213717693836978,3,0,0,True
|
||||||
|
simple_qa_00025,simple_qa,answering,1219.6170000000031,4464,2084,2.1420345489443378,3,0,0,True
|
||||||
|
simple_qa_00026,simple_qa,answering,753.5990000000013,4068,1755,2.317948717948718,3,0,0,True
|
||||||
|
simple_qa_00027,simple_qa,answering,791.2379999999998,5056,1373,3.6824471959213403,3,0,0,True
|
||||||
|
simple_qa_00028,simple_qa,answering,827.5049999999986,3795,801,4.737827715355805,3,0,0,True
|
||||||
|
simple_qa_00029,simple_qa,answering,677.6099999999979,4655,2300,2.023913043478261,3,0,0,True
|
||||||
|
simple_qa_00030,simple_qa,answering,791.8140000000022,4267,720,5.926388888888889,3,0,0,True
|
||||||
|
simple_qa_00031,simple_qa,answering,1215.0280000000002,7292,3124,2.3341869398207424,5,1,0,True
|
||||||
|
simple_qa_00032,simple_qa,answering,1804.2910000000027,4607,2837,1.6238984843144166,3,0,0,True
|
||||||
|
simple_qa_00033,simple_qa,answering,844.0370000000001,3221,1756,1.8342824601366743,3,0,0,True
|
||||||
|
simple_qa_00034,simple_qa,answering,1057.2420000000022,5454,1504,3.6263297872340425,3,0,0,True
|
||||||
|
simple_qa_00035,simple_qa,answering,1576.5220000000006,5750,1500,3.8333333333333335,3,0,0,True
|
||||||
|
simple_qa_00036,simple_qa,answering,1026.4500000000005,7409,4583,1.616626663757364,3,0,0,True
|
||||||
|
simple_qa_00037,simple_qa,answering,716.7670000000008,5122,1777,2.8823860438942037,3,0,0,True
|
||||||
|
simple_qa_00038,simple_qa,answering,982.6349999999984,7826,1145,6.834934497816594,3,0,0,True
|
||||||
|
simple_qa_00039,simple_qa,answering,701.6949999999973,5078,670,7.57910447761194,3,0,0,True
|
||||||
|
simple_qa_00040,simple_qa,answering,1318.7110000000005,5824,2164,2.6913123844731976,3,0,0,True
|
||||||
|
simple_qa_00041,simple_qa,answering,839.9459999999976,6330,1513,4.183740912095175,3,0,0,True
|
||||||
|
simple_qa_00042,simple_qa,answering,1454.4789999999991,5313,1358,3.9123711340206184,3,0,0,True
|
||||||
|
simple_qa_00043,simple_qa,answering,1366.8249999999987,5530,733,7.544338335607094,3,0,0,True
|
||||||
|
simple_qa_00044,simple_qa,answering,1134.8300000000008,5574,919,6.065288356909685,3,0,0,True
|
||||||
|
simple_qa_00045,simple_qa,answering,1290.6880000000028,8607,1321,6.51551854655564,5,1,0,True
|
||||||
|
simple_qa_00046,simple_qa,answering,887.5910000000005,5342,1220,4.378688524590164,3,0,0,True
|
||||||
|
simple_qa_00047,simple_qa,answering,1500.1400000000017,11544,1496,7.716577540106952,5,0,0,True
|
||||||
|
simple_qa_00048,simple_qa,answering,590.2030000000025,3429,1270,2.7,3,0,0,True
|
||||||
|
simple_qa_00049,simple_qa,answering,1328.8580000000038,3926,1557,2.5215157353885678,3,0,0,True
|
||||||
|
simple_qa_00050,simple_qa,answering,895.7130000000006,4332,1319,3.284306292645944,3,0,0,True
|
||||||
|
simple_qa_00051,simple_qa,answering,754.6930000000032,6203,1342,4.62220566318927,3,0,0,True
|
||||||
|
simple_qa_00052,simple_qa,answering,1541.4049999999975,5071,1117,4.539838854073411,3,0,0,True
|
||||||
|
simple_qa_00053,simple_qa,answering,1177.0819999999985,8016,1230,6.517073170731707,5,1,0,True
|
||||||
|
simple_qa_00054,simple_qa,answering,498.7519999999961,5493,391,14.048593350383632,3,0,0,True
|
||||||
|
simple_qa_00055,simple_qa,answering,1162.9900000000007,7303,1651,4.423379769836463,3,0,0,True
|
||||||
|
simple_qa_00056,simple_qa,answering,674.510000000005,2802,669,4.188340807174888,3,0,0,True
|
||||||
|
simple_qa_00057,simple_qa,answering,2256.7659999999987,4338,969,4.476780185758514,3,0,0,True
|
||||||
|
simple_qa_00058,simple_qa,answering,1575.8580000000038,10725,3214,3.336963285625389,5,1,0,True
|
||||||
|
simple_qa_00059,simple_qa,answering,1815.6789999999958,5774,1265,4.564426877470356,3,0,0,True
|
||||||
|
simple_qa_00060,simple_qa,answering,686.0010000000045,5624,923,6.0931744312026,3,0,0,True
|
||||||
|
simple_qa_00061,simple_qa,answering,1254.2200000000037,6000,887,6.764374295377678,3,0,0,True
|
||||||
|
simple_qa_00062,simple_qa,answering,945.3009999999936,7475,564,13.25354609929078,3,0,0,True
|
||||||
|
simple_qa_00063,simple_qa,answering,841.0459999999987,3683,751,4.9041278295605855,3,0,0,True
|
||||||
|
simple_qa_00064,simple_qa,answering,1241.6660000000022,9729,904,10.76216814159292,5,1,0,True
|
||||||
|
simple_qa_00065,simple_qa,answering,1516.855999999997,6629,1109,5.977457168620378,5,1,0,True
|
||||||
|
simple_qa_00066,simple_qa,answering,1183.698999999997,3552,1627,2.183159188690842,3,0,0,True
|
||||||
|
simple_qa_00067,simple_qa,answering,995.9650000000054,8377,2106,3.977682811016144,3,0,0,True
|
||||||
|
simple_qa_00068,simple_qa,answering,959.3050000000005,4731,1465,3.2293515358361775,3,0,0,True
|
||||||
|
simple_qa_00069,simple_qa,answering,1188.1830000000023,3612,1113,3.2452830188679247,3,0,0,True
|
||||||
|
simple_qa_00070,simple_qa,answering,1491.27,4065,761,5.341655716162943,3,0,0,True
|
||||||
|
simple_qa_00071,simple_qa,answering,846.0699999999974,2898,1350,2.1466666666666665,3,0,0,True
|
||||||
|
simple_qa_00072,simple_qa,answering,1207.8729999999994,3417,747,4.57429718875502,3,0,0,True
|
||||||
|
simple_qa_00073,simple_qa,answering,1940.8279999999963,7028,3264,2.153186274509804,5,1,0,True
|
||||||
|
simple_qa_00074,simple_qa,answering,1482.2769999999964,5076,1081,4.695652173913044,3,0,0,True
|
||||||
|
simple_qa_00075,simple_qa,answering,864.4090000000019,6816,897,7.59866220735786,3,0,0,True
|
||||||
|
simple_qa_00076,simple_qa,answering,1091.8100000000024,4841,1310,3.6954198473282442,3,0,0,True
|
||||||
|
simple_qa_00077,simple_qa,answering,1149.9900000000025,4287,737,5.8168249660786975,3,0,0,True
|
||||||
|
simple_qa_00078,simple_qa,answering,1253.6500000000003,6102,1037,5.884281581485053,3,0,0,True
|
||||||
|
simple_qa_00079,simple_qa,answering,2618.4089999999997,6982,680,10.26764705882353,5,1,0,True
|
||||||
|
simple_qa_00080,simple_qa,answering,838.6209999999962,5148,824,6.247572815533981,3,0,0,True
|
||||||
|
simple_qa_00081,simple_qa,answering,706.4880000000003,8520,643,13.250388802488336,3,0,0,True
|
||||||
|
simple_qa_00082,simple_qa,answering,1185.3399999999965,4372,474,9.223628691983123,3,0,0,True
|
||||||
|
simple_qa_00083,simple_qa,answering,1010.4350000000011,8796,1347,6.5300668151447665,3,0,0,True
|
||||||
|
simple_qa_00084,simple_qa,answering,652.358999999997,3394,1270,2.6724409448818895,3,0,0,True
|
||||||
|
simple_qa_00085,simple_qa,answering,1272.1099999999979,5066,1257,4.030230708035004,3,0,0,True
|
||||||
|
simple_qa_00086,simple_qa,answering,1247.2220000000007,7260,610,11.901639344262295,5,1,0,True
|
||||||
|
simple_qa_00087,simple_qa,answering,813.3010000000027,3990,2088,1.910919540229885,3,0,0,True
|
||||||
|
simple_qa_00088,simple_qa,answering,1376.7839999999935,2576,1292,1.9938080495356036,3,0,0,True
|
||||||
|
simple_qa_00089,simple_qa,answering,727.212999999999,5616,1291,4.350116189000775,3,0,0,True
|
||||||
|
simple_qa_00090,simple_qa,answering,1237.8660000000039,7402,851,8.698002350176264,3,0,0,True
|
||||||
|
simple_qa_00091,simple_qa,answering,757.8150000000007,9526,1514,6.291941875825628,5,1,0,True
|
||||||
|
simple_qa_00092,simple_qa,answering,2362.844000000003,4959,1026,4.833333333333333,3,0,0,True
|
||||||
|
simple_qa_00093,simple_qa,answering,1152.037,4596,920,4.995652173913044,3,0,0,True
|
||||||
|
simple_qa_00094,simple_qa,answering,1841.4469999999951,4404,1581,2.7855787476280836,3,0,0,True
|
||||||
|
simple_qa_00095,simple_qa,answering,1221.4370000000017,5886,1182,4.979695431472082,3,0,0,True
|
||||||
|
simple_qa_00096,simple_qa,answering,1167.4190000000024,5641,2854,1.9765241765942536,3,0,0,True
|
||||||
|
simple_qa_00097,simple_qa,answering,1575.9909999999948,4913,1439,3.414176511466296,3,0,0,True
|
||||||
|
simple_qa_00098,simple_qa,answering,1244.641999999999,8507,1101,7.72661217075386,3,0,0,True
|
||||||
|
simple_qa_00099,simple_qa,answering,1048.887999999998,4661,1839,2.5345296356715608,3,0,0,True
|
||||||
|
tool_research_00000,tool_research,evidence_collection,2194.9050000000057,24867,3202,7.766083697688944,5,1,0,True
|
||||||
|
tool_research_00001,tool_research,evidence_collection,3702.2710000000034,43758,1917,22.826291079812208,9,2,0,True
|
||||||
|
tool_research_00002,tool_research,evidence_collection,4914.04399999999,29867,1906,15.669989506820567,5,1,0,True
|
||||||
|
tool_research_00003,tool_research,evidence_collection,2975.851999999996,18814,4275,4.40093567251462,5,1,0,True
|
||||||
|
tool_research_00004,tool_research,evidence_collection,1859.4280000000012,17926,1263,14.193190815518607,3,0,0,True
|
||||||
|
tool_research_00005,tool_research,evidence_collection,1666.4189999999976,13686,8729,1.56787719097262,5,1,0,True
|
||||||
|
tool_research_00006,tool_research,evidence_collection,2696.655000000007,9331,1703,5.479154433352907,3,0,0,True
|
||||||
|
tool_research_00007,tool_research,evidence_collection,5450.783000000001,13849,1654,8.373035066505441,6,2,1,True
|
||||||
|
tool_research_00008,tool_research,evidence_collection,1531.513000000004,17238,3296,5.22997572815534,5,1,0,True
|
||||||
|
tool_research_00009,tool_research,evidence_collection,4081.8340000000007,13926,436,31.940366972477065,5,1,0,True
|
||||||
|
tool_research_00010,tool_research,evidence_collection,4327.119000000011,41953,2235,18.770917225950782,5,1,0,True
|
||||||
|
tool_research_00011,tool_research,evidence_collection,3781.2230000000113,15091,3794,3.977596204533474,5,1,0,True
|
||||||
|
tool_research_00012,tool_research,evidence_collection,1917.4310000000078,14626,1784,8.198430493273543,3,0,0,True
|
||||||
|
tool_research_00013,tool_research,evidence_collection,3997.425000000007,13485,1603,8.412351840299438,5,1,0,True
|
||||||
|
tool_research_00014,tool_research,evidence_collection,2441.9809999999984,36453,2047,17.80801172447484,5,1,0,True
|
||||||
|
tool_research_00015,tool_research,evidence_collection,3959.5280000000057,15873,1390,11.419424460431655,3,0,0,True
|
||||||
|
tool_research_00016,tool_research,evidence_collection,1198.003,13816,976,14.155737704918034,3,0,0,True
|
||||||
|
tool_research_00017,tool_research,evidence_collection,4552.255999999999,16333,2462,6.634037367993502,5,1,0,True
|
||||||
|
tool_research_00018,tool_research,evidence_collection,6950.037999999992,39260,5865,6.693947144075021,6,2,1,True
|
||||||
|
tool_research_00019,tool_research,evidence_collection,3067.522999999994,15009,2037,7.368188512518409,3,0,0,True
|
||||||
|
tool_research_00020,tool_research,evidence_collection,4835.037999999997,30787,1662,18.524067388688326,7,1,0,True
|
||||||
|
tool_research_00021,tool_research,evidence_collection,8310.215999999997,113647,4305,26.39883855981417,17,4,0,True
|
||||||
|
tool_research_00022,tool_research,evidence_collection,3200.504999999993,18179,1048,17.346374045801525,5,1,0,True
|
||||||
|
tool_research_00023,tool_research,evidence_collection,13355.828000000003,126809,1205,105.2356846473029,15,5,1,True
|
||||||
|
tool_research_00024,tool_research,evidence_collection,3643.411999999998,21779,2025,10.755061728395061,5,1,0,True
|
||||||
|
tool_research_00025,tool_research,evidence_collection,9670.201000000006,47663,1365,34.91794871794872,13,4,2,True
|
||||||
|
tool_research_00026,tool_research,evidence_collection,1291.9939999999883,12623,3997,3.1581185889417065,3,0,0,True
|
||||||
|
tool_research_00027,tool_research,evidence_collection,4673.997999999998,27499,1513,18.17514871116986,5,1,0,True
|
||||||
|
tool_research_00028,tool_research,evidence_collection,2161.7999999999993,32414,1401,23.13633119200571,5,1,0,True
|
||||||
|
tool_research_00029,tool_research,evidence_collection,5903.75499999999,21437,3963,5.40928589452435,5,1,0,True
|
||||||
|
tool_research_00030,tool_research,evidence_collection,4832.2890000000025,62170,2517,24.70003972983711,9,2,0,True
|
||||||
|
tool_research_00031,tool_research,evidence_collection,4330.880000000007,29507,2884,10.23127600554785,6,2,1,True
|
||||||
|
tool_research_00032,tool_research,evidence_collection,1570.5789999999952,18174,849,21.406360424028268,5,1,0,True
|
||||||
|
tool_research_00033,tool_research,evidence_collection,1797.4840000000113,34726,2226,15.600179694519317,5,1,0,True
|
||||||
|
tool_research_00034,tool_research,evidence_collection,3040.871999999993,19123,2094,9.13228271251194,5,1,0,True
|
||||||
|
tool_research_00035,tool_research,evidence_collection,2986.626000000001,24572,1303,18.858019953952418,5,1,0,True
|
||||||
|
tool_research_00036,tool_research,evidence_collection,3130.6490000000053,28061,1577,17.793912492073556,3,0,0,True
|
||||||
|
tool_research_00037,tool_research,evidence_collection,3123.7370000000055,22728,3665,6.201364256480218,5,1,0,True
|
||||||
|
tool_research_00038,tool_research,evidence_collection,1708.3070000000048,29910,1078,27.74582560296846,5,1,0,True
|
||||||
|
tool_research_00039,tool_research,evidence_collection,4255.195999999998,21270,1412,15.063739376770538,5,1,0,True
|
||||||
|
tool_research_00040,tool_research,evidence_collection,2188.941,24341,4087,5.955713237093223,5,1,0,True
|
||||||
|
tool_research_00041,tool_research,evidence_collection,2437.484999999995,4248,1853,2.2924986508364813,3,0,0,True
|
||||||
|
tool_research_00042,tool_research,evidence_collection,2187.698999999995,8817,2395,3.681419624217119,3,0,0,True
|
||||||
|
tool_research_00043,tool_research,evidence_collection,5124.787999999995,36771,2156,17.055194805194805,9,2,0,True
|
||||||
|
tool_research_00044,tool_research,evidence_collection,2136.277000000007,23014,3951,5.824854467223488,5,1,0,True
|
||||||
|
tool_research_00045,tool_research,evidence_collection,2712.2329999999974,13715,4904,2.7966965742251224,5,1,0,True
|
||||||
|
tool_research_00046,tool_research,evidence_collection,2093.755999999999,27149,2497,10.872647176611935,3,0,0,True
|
||||||
|
tool_research_00047,tool_research,evidence_collection,3331.8359999999957,10694,2450,4.364897959183674,3,0,0,True
|
||||||
|
tool_research_00048,tool_research,evidence_collection,2496.739000000005,23797,420,56.65952380952381,5,1,0,True
|
||||||
|
tool_research_00049,tool_research,evidence_collection,3162.072999999992,37174,6914,5.376627133352618,5,1,0,True
|
||||||
|
tool_research_00050,tool_research,evidence_collection,3349.9610000000075,20453,1503,13.608117099135063,5,1,0,True
|
||||||
|
tool_research_00051,tool_research,evidence_collection,2585.735999999997,20752,4970,4.175452716297786,5,1,0,True
|
||||||
|
tool_research_00052,tool_research,evidence_collection,2241.700999999992,18226,1014,17.974358974358974,5,1,0,True
|
||||||
|
tool_research_00053,tool_research,evidence_collection,1681.621000000007,20527,1964,10.451629327902241,5,1,0,True
|
||||||
|
tool_research_00054,tool_research,evidence_collection,4616.5809999999965,14793,1020,14.50294117647059,5,1,0,True
|
||||||
|
tool_research_00055,tool_research,evidence_collection,1986.9599999999964,36674,1499,24.46564376250834,5,1,0,True
|
||||||
|
tool_research_00056,tool_research,evidence_collection,3329.991000000007,21611,2138,10.108044901777362,5,1,0,True
|
||||||
|
tool_research_00057,tool_research,evidence_collection,1935.023000000001,11508,3856,2.9844398340248963,3,0,0,True
|
||||||
|
tool_research_00058,tool_research,evidence_collection,2901.527999999999,31197,1663,18.759470835838844,5,1,0,True
|
||||||
|
tool_research_00059,tool_research,evidence_collection,3321.638000000007,11370,816,13.933823529411764,3,0,0,True
|
||||||
|
tool_research_00060,tool_research,evidence_collection,3683.4269999999947,22492,1908,11.78825995807128,5,1,0,True
|
||||||
|
tool_research_00061,tool_research,evidence_collection,2848.731999999998,18988,1760,10.788636363636364,5,1,0,True
|
||||||
|
tool_research_00062,tool_research,evidence_collection,2672.6560000000036,11461,1837,6.238976592270006,5,1,0,True
|
||||||
|
tool_research_00063,tool_research,evidence_collection,2669.6290000000004,12186,2987,4.079678607298293,3,0,0,True
|
||||||
|
tool_research_00064,tool_research,evidence_collection,1733.4749999999985,20624,2281,9.041648399824638,5,1,0,True
|
||||||
|
tool_research_00065,tool_research,evidence_collection,1969.7800000000002,37289,2064,18.066375968992247,5,1,0,True
|
||||||
|
tool_research_00066,tool_research,evidence_collection,2515.674000000004,27763,948,29.285864978902953,5,1,0,True
|
||||||
|
tool_research_00067,tool_research,evidence_collection,3096.862999999999,25812,1096,23.55109489051095,5,1,0,True
|
||||||
|
tool_research_00068,tool_research,evidence_collection,2049.244999999999,27451,1646,16.677399756986635,5,1,0,True
|
||||||
|
tool_research_00069,tool_research,evidence_collection,1731.0909999999922,25997,2954,8.800609343263371,5,1,0,True
|
||||||
|
tool_research_00070,tool_research,evidence_collection,3312.636999999995,31567,3135,10.069218500797447,5,1,0,True
|
||||||
|
tool_research_00071,tool_research,evidence_collection,1862.328000000005,20908,2780,7.520863309352518,5,1,0,True
|
||||||
|
tool_research_00072,tool_research,evidence_collection,3047.896000000009,54132,1025,52.81170731707317,5,1,0,True
|
||||||
|
tool_research_00073,tool_research,evidence_collection,2561.8630000000026,21435,4790,4.474947807933194,5,1,0,True
|
||||||
|
tool_research_00074,tool_research,evidence_collection,4560.357999999994,79773,2385,33.44779874213837,13,3,0,True
|
||||||
|
tool_research_00075,tool_research,evidence_collection,3243.395000000007,7843,2993,2.6204477113264284,3,0,0,True
|
||||||
|
tool_research_00076,tool_research,evidence_collection,2738.9289999999987,23031,2795,8.240071556350626,5,1,0,True
|
||||||
|
tool_research_00077,tool_research,evidence_collection,5143.501999999998,26921,3689,7.297641637300082,3,0,0,True
|
||||||
|
tool_research_00078,tool_research,evidence_collection,2541.331999999997,10770,5383,2.000743080066877,3,0,0,True
|
||||||
|
tool_research_00079,tool_research,evidence_collection,4596.682000000002,30560,3128,9.769820971867007,9,2,0,True
|
||||||
|
tool_research_00080,tool_research,evidence_collection,2516.4350000000013,10007,1693,5.910809214412286,3,0,0,True
|
||||||
|
tool_research_00081,tool_research,evidence_collection,3084.7260000000033,17944,830,21.619277108433735,5,1,0,True
|
||||||
|
tool_research_00082,tool_research,evidence_collection,2914.749999999998,25400,982,25.865580448065174,5,1,0,True
|
||||||
|
tool_research_00083,tool_research,evidence_collection,2230.4889999999914,24604,2144,11.475746268656716,5,1,0,True
|
||||||
|
tool_research_00084,tool_research,evidence_collection,3443.204000000009,24850,739,33.62652232746955,5,1,0,True
|
||||||
|
tool_research_00085,tool_research,evidence_collection,5764.497000000006,31697,6531,4.853314959424284,6,2,1,True
|
||||||
|
tool_research_00086,tool_research,evidence_collection,2663.731999999996,14968,967,15.478800413650465,5,1,0,True
|
||||||
|
tool_research_00087,tool_research,evidence_collection,1660.2049999999906,10680,3250,3.286153846153846,5,1,0,True
|
||||||
|
tool_research_00088,tool_research,evidence_collection,1999.1580000000085,11601,3306,3.5090744101633393,3,0,0,True
|
||||||
|
tool_research_00089,tool_research,evidence_collection,2695.4450000000065,10711,3265,3.280551301684533,5,1,0,True
|
||||||
|
tool_research_00090,tool_research,evidence_collection,5276.9389999999985,15851,1980,8.005555555555556,5,1,0,True
|
||||||
|
tool_research_00091,tool_research,evidence_collection,5764.26699999999,27062,2151,12.581125058112505,6,2,1,True
|
||||||
|
tool_research_00092,tool_research,evidence_collection,1482.0979999999936,17916,936,19.141025641025642,5,1,0,True
|
||||||
|
tool_research_00093,tool_research,evidence_collection,2160.2140000000104,36749,2052,17.908869395711502,5,1,0,True
|
||||||
|
tool_research_00094,tool_research,evidence_collection,3420.692000000017,15674,8252,1.8994183228308288,3,0,0,True
|
||||||
|
tool_research_00095,tool_research,evidence_collection,1882.05099999999,19776,2123,9.315120113047573,5,1,0,True
|
||||||
|
tool_research_00096,tool_research,evidence_collection,3586.549000000005,36710,1309,28.044308632543927,5,1,0,True
|
||||||
|
tool_research_00097,tool_research,evidence_collection,1987.364999999997,14435,3124,4.62067861715749,5,1,0,True
|
||||||
|
tool_research_00098,tool_research,evidence_collection,3917.211000000009,16232,1378,11.779390420899855,6,2,1,True
|
||||||
|
tool_research_00099,tool_research,evidence_collection,4009.145999999987,74407,2235,33.2917225950783,5,1,0,True
|
||||||
|
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
Binary file not shown.
|
After Width: | Height: | Size: 46 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 112 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 82 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 145 KiB |
@@ -0,0 +1,183 @@
|
|||||||
|
{
|
||||||
|
"global": {
|
||||||
|
"transition_probabilities": {
|
||||||
|
"CallTool->Think": 0.9891472868217054,
|
||||||
|
"Idle->Think": 0.9911167512690355,
|
||||||
|
"Retry->CallTool": 0.9176470588235294,
|
||||||
|
"Think->CallAgent": 0.12293086660175268,
|
||||||
|
"Think->CallTool": 0.15603700097370984,
|
||||||
|
"Think->Failed": 0.0017039922103213243,
|
||||||
|
"Think->Send": 0.5148490749756572,
|
||||||
|
"Think->Split": 0.20374878286270692,
|
||||||
|
"Wait->Failed": 0.36363636363636365
|
||||||
|
},
|
||||||
|
"duration_ms": {
|
||||||
|
"mean": 285.679917634447,
|
||||||
|
"std": 550.6643665581886,
|
||||||
|
"cv": 1.9275571454862042,
|
||||||
|
"p50": 8.0,
|
||||||
|
"p95": 1323.2897999999977,
|
||||||
|
"count": 5597
|
||||||
|
},
|
||||||
|
"message_size_bytes": {
|
||||||
|
"mean": 5570.409505092013,
|
||||||
|
"std": 6586.248171004335,
|
||||||
|
"cv": 1.1823633729232519,
|
||||||
|
"p50": 3158.0,
|
||||||
|
"p95": 18393.199999999997,
|
||||||
|
"count": 5597
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"task_types": {
|
||||||
|
"collaborative_analysis": {
|
||||||
|
"phase": "multi_agent_synthesis",
|
||||||
|
"task_count": 260,
|
||||||
|
"tool_probability": 0.4806201550387597,
|
||||||
|
"split_probability": 0.8192307692307692,
|
||||||
|
"mean_subtasks": 2.976923076923077,
|
||||||
|
"retry_probability": 0.12470588235294118,
|
||||||
|
"mean_retries_if_any": 1.1627906976744187,
|
||||||
|
"failure_probability": 0.007692307692307693,
|
||||||
|
"think_time_ms": {
|
||||||
|
"mean": 1012.4040078828829,
|
||||||
|
"std": 819.189229985454,
|
||||||
|
"cv": 0.8091524960460444,
|
||||||
|
"p50": 802.0150000000001,
|
||||||
|
"p95": 2609.3335999999995,
|
||||||
|
"count": 888
|
||||||
|
},
|
||||||
|
"external_size_bytes": {
|
||||||
|
"mean": 4103.692307692308,
|
||||||
|
"std": 3820.340621837784,
|
||||||
|
"cv": 0.9309520147688984,
|
||||||
|
"p50": 3039.5,
|
||||||
|
"p95": 11107.699999999995,
|
||||||
|
"count": 260
|
||||||
|
},
|
||||||
|
"request_size_bytes": {
|
||||||
|
"mean": 3936.623019182652,
|
||||||
|
"std": 3891.679972776879,
|
||||||
|
"cv": 0.9885833501997089,
|
||||||
|
"p50": 2796.0,
|
||||||
|
"p95": 10800.3,
|
||||||
|
"count": 1199
|
||||||
|
},
|
||||||
|
"response_size_bytes": {
|
||||||
|
"mean": 11829.127142857144,
|
||||||
|
"std": 8952.816088861513,
|
||||||
|
"cv": 0.7568450301312002,
|
||||||
|
"p50": 9558.5,
|
||||||
|
"p95": 29122.249999999993,
|
||||||
|
"count": 1400
|
||||||
|
},
|
||||||
|
"message_count_per_task": {
|
||||||
|
"mean": 11.01923076923077,
|
||||||
|
"std": 5.376910696585285,
|
||||||
|
"cv": 0.48795699166218987,
|
||||||
|
"p50": 10.0,
|
||||||
|
"p95": 21.049999999999983,
|
||||||
|
"count": 260
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"simple_qa": {
|
||||||
|
"phase": "answering",
|
||||||
|
"task_count": 260,
|
||||||
|
"tool_probability": 0.13740458015267176,
|
||||||
|
"split_probability": 0.03461538461538462,
|
||||||
|
"mean_subtasks": 1.0076923076923077,
|
||||||
|
"retry_probability": 0.02702702702702703,
|
||||||
|
"mean_retries_if_any": 1.0,
|
||||||
|
"failure_probability": 0.0,
|
||||||
|
"think_time_ms": {
|
||||||
|
"mean": 315.74304316546767,
|
||||||
|
"std": 193.59344702605458,
|
||||||
|
"cv": 0.6131360649634342,
|
||||||
|
"p50": 262.062,
|
||||||
|
"p95": 693.3605,
|
||||||
|
"count": 556
|
||||||
|
},
|
||||||
|
"external_size_bytes": {
|
||||||
|
"mean": 1285.7846153846153,
|
||||||
|
"std": 562.841233039538,
|
||||||
|
"cv": 0.43774145864327046,
|
||||||
|
"p50": 1167.5,
|
||||||
|
"p95": 2332.1,
|
||||||
|
"count": 260
|
||||||
|
},
|
||||||
|
"request_size_bytes": {
|
||||||
|
"mean": 1646.314381270903,
|
||||||
|
"std": 819.0778288985601,
|
||||||
|
"cv": 0.4975221247027301,
|
||||||
|
"p50": 1537.0,
|
||||||
|
"p95": 3065.4999999999995,
|
||||||
|
"count": 299
|
||||||
|
},
|
||||||
|
"response_size_bytes": {
|
||||||
|
"mean": 1909.388888888889,
|
||||||
|
"std": 1208.6473367395993,
|
||||||
|
"cv": 0.6330021839830309,
|
||||||
|
"p50": 1577.5,
|
||||||
|
"p95": 4346.099999999999,
|
||||||
|
"count": 558
|
||||||
|
},
|
||||||
|
"message_count_per_task": {
|
||||||
|
"mean": 4.296153846153846,
|
||||||
|
"std": 0.7184281823915581,
|
||||||
|
"cv": 0.16722589742328123,
|
||||||
|
"p50": 4.0,
|
||||||
|
"p95": 6.0,
|
||||||
|
"count": 260
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"tool_research": {
|
||||||
|
"phase": "evidence_collection",
|
||||||
|
"task_count": 260,
|
||||||
|
"tool_probability": 0.7631578947368421,
|
||||||
|
"split_probability": 0.2076923076923077,
|
||||||
|
"mean_subtasks": 1.1692307692307693,
|
||||||
|
"retry_probability": 0.09019607843137255,
|
||||||
|
"mean_retries_if_any": 1.0952380952380953,
|
||||||
|
"failure_probability": 0.0038461538461538464,
|
||||||
|
"think_time_ms": {
|
||||||
|
"mean": 660.7411426666666,
|
||||||
|
"std": 548.0475005635645,
|
||||||
|
"cv": 0.8294435826286146,
|
||||||
|
"p50": 513.7415,
|
||||||
|
"p95": 1615.4994999999997,
|
||||||
|
"count": 750
|
||||||
|
},
|
||||||
|
"external_size_bytes": {
|
||||||
|
"mean": 2591.223076923077,
|
||||||
|
"std": 1938.9205968801086,
|
||||||
|
"cv": 0.7482646377101818,
|
||||||
|
"p50": 2023.0,
|
||||||
|
"p95": 6498.299999999999,
|
||||||
|
"count": 260
|
||||||
|
},
|
||||||
|
"request_size_bytes": {
|
||||||
|
"mean": 2178.8425760286227,
|
||||||
|
"std": 1869.411620553448,
|
||||||
|
"cv": 0.8579837942954215,
|
||||||
|
"p50": 1586.0,
|
||||||
|
"p95": 6085.500000000001,
|
||||||
|
"count": 559
|
||||||
|
},
|
||||||
|
"response_size_bytes": {
|
||||||
|
"mean": 6327.029003783102,
|
||||||
|
"std": 4973.701881543845,
|
||||||
|
"cv": 0.7861038535732859,
|
||||||
|
"p50": 4987.0,
|
||||||
|
"p95": 15569.8,
|
||||||
|
"count": 793
|
||||||
|
},
|
||||||
|
"message_count_per_task": {
|
||||||
|
"mean": 6.211538461538462,
|
||||||
|
"std": 2.097035805988451,
|
||||||
|
"cv": 0.33760328765139147,
|
||||||
|
"p50": 6.0,
|
||||||
|
"p95": 11.0,
|
||||||
|
"count": 260
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
timeout_probability,max_retries,success_rate,mean_retries,mean_amplification,mean_latency_ms
|
||||||
|
0.0,0,1.0,0.0,9.014727352307537,1829.8204232372348
|
||||||
|
0.0,1,1.0,0.0,9.017931822157774,1786.1398565415025
|
||||||
|
0.0,2,1.0,0.0,9.497944538818517,1655.013246029438
|
||||||
|
0.0,3,1.0,0.0,8.981130309004985,1716.8465797602803
|
||||||
|
0.0,5,1.0,0.0,9.540066677486474,1716.7430554392593
|
||||||
|
0.03,0,0.9666666666666667,0.0,9.063950194066456,1669.7925665017842
|
||||||
|
0.03,1,0.9958333333333333,0.016666666666666666,9.063781276545864,1657.0420076151597
|
||||||
|
0.03,2,1.0,0.016666666666666666,9.014566351856745,1691.3863870856096
|
||||||
|
0.03,3,1.0,0.020833333333333332,9.637237082920953,1721.830506046892
|
||||||
|
0.03,5,1.0,0.0125,9.649347558637414,1718.5997383083138
|
||||||
|
0.08,0,0.9041666666666667,0.0,8.25627307377185,1635.0229558657152
|
||||||
|
0.08,1,1.0,0.05416666666666667,9.289215941037927,1787.9821751080794
|
||||||
|
0.08,2,1.0,0.06666666666666667,9.431797013914702,1916.4375526605215
|
||||||
|
0.08,3,1.0,0.058333333333333334,9.050271941109205,1898.8272546658047
|
||||||
|
0.08,5,1.0,0.07916666666666666,9.741006123762736,1970.2545335768014
|
||||||
|
0.15,0,0.8416666666666667,0.0,8.164931539209656,1610.0725491672833
|
||||||
|
0.15,1,0.9833333333333333,0.10833333333333334,9.088215821284614,1938.8382504596384
|
||||||
|
0.15,2,1.0,0.1,9.261076832398844,1941.657277871303
|
||||||
|
0.15,3,0.9958333333333333,0.13333333333333333,11.372866350773522,2027.2488180006546
|
||||||
|
0.15,5,1.0,0.13333333333333333,8.152159646681561,2064.0974036821353
|
||||||
|
0.25,0,0.7916666666666666,0.0,7.82021056651408,1652.063945123461
|
||||||
|
0.25,1,0.9541666666666667,0.17916666666666667,8.424662742312817,2194.736913880946
|
||||||
|
0.25,2,0.9916666666666667,0.2875,9.275168718803195,2475.475754982108
|
||||||
|
0.25,3,1.0,0.2833333333333333,8.808713566195896,2429.5919980366966
|
||||||
|
0.25,5,1.0,0.1875,9.459047702518982,2044.5400456955706
|
||||||
|
@@ -0,0 +1,9 @@
|
|||||||
|
arrival_rate,offered_tasks,completed_tasks,throughput_per_second,mean_delay_ms,p95_delay_ms,queue_peak,drop_rate,success_rate,mean_amplification
|
||||||
|
0.5,87,79,0.49375,5690.601711699437,12756.456777674,2,0.0,1.0,32.68982726898488
|
||||||
|
1.0,178,154,0.9625,28654.713037581158,50223.56851458479,35,0.0,1.0,35.61819225706701
|
||||||
|
2.0,354,315,1.96875,159265.0479860076,259462.94208402873,216,0.0,1.0,27.685762197623816
|
||||||
|
4.0,735,553,3.45625,360462.8938916123,611043.0735286651,500,0.1251700680272109,0.9981916817359855,28.819579849617657
|
||||||
|
6.0,1140,538,3.3625,406978.074020204,627164.1820327837,500,0.4280701754385965,1.0,30.664346517813648
|
||||||
|
8.0,1419,478,2.9875,439078.312694735,605565.2288237411,500,0.5419309372797745,0.997907949790795,28.477068296105013
|
||||||
|
10.0,1736,459,2.86875,462378.903316947,623226.2655493785,500,0.6215437788018433,0.9978213507625272,29.655131218317656
|
||||||
|
12.0,2196,407,2.54375,503595.3336070842,641094.366587491,500,0.7108378870673953,1.0,29.595658555137497
|
||||||
|
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"best_model_by_internal_bytes_mape": {
|
||||||
|
"model": "full_model",
|
||||||
|
"metric": "internal_bytes",
|
||||||
|
"mae": 9852.85421644164,
|
||||||
|
"rmse": 11957.784318614104,
|
||||||
|
"mape_percent": 27.25831155178904
|
||||||
|
},
|
||||||
|
"best_model_by_latency_mape": {
|
||||||
|
"model": "full_model",
|
||||||
|
"metric": "latency_ms",
|
||||||
|
"mae": 1677.2186263904505,
|
||||||
|
"rmse": 2068.228406399321,
|
||||||
|
"mape_percent": 38.01609858429338
|
||||||
|
},
|
||||||
|
"max_stress_arrival_rate": 12.0,
|
||||||
|
"max_observed_p95_delay_ms": 641094.366587491
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
model,metric,mae,rmse,mape_percent
|
||||||
|
full_model,internal_bytes,9852.85421644164,11957.784318614104,27.25831155178904
|
||||||
|
full_model,latency_ms,1677.2186263904505,2068.228406399321,38.01609858429338
|
||||||
|
full_model,message_count,0.43277777777777793,0.5471686380984558,5.855207621776822
|
||||||
|
full_model,tool_calls,0.11166666666666662,0.14041275134666625,14.427527090058401
|
||||||
|
full_model,amplification,4.305062499013978,5.243570352907988,22.67666226433828
|
||||||
|
no_context,internal_bytes,28387.670805477606,38228.5095522755,119.54177279524332
|
||||||
|
no_context,latency_ms,2907.231795727213,3869.904048337353,62.66674153347929
|
||||||
|
no_context,message_count,2.485555555555556,3.1931712438565825,38.768826743972944
|
||||||
|
no_context,tool_calls,0.6505555555555556,0.7056603159321639,196.9584307828373
|
||||||
|
no_context,amplification,10.922275115304535,13.789521598016519,70.65863904754978
|
||||||
|
static_mean,internal_bytes,29834.93333333333,32709.783175317236,230.39064998346973
|
||||||
|
static_mean,latency_ms,2757.6886622222223,3058.6901465213614,119.58343818448623
|
||||||
|
static_mean,message_count,2.5155555555555558,2.7855500155066126,48.03496553983882
|
||||||
|
static_mean,tool_calls,0.5444444444444444,0.6355225321645873,264.95059144468667
|
||||||
|
static_mean,amplification,10.71876769134868,12.026846682885607,111.10365289444462
|
||||||
|
@@ -0,0 +1,10 @@
|
|||||||
|
task_type,model,actual_internal_bytes,predicted_internal_bytes,actual_latency_ms,predicted_latency_ms,actual_message_count,predicted_message_count,actual_tool_calls,predicted_tool_calls,actual_amplification,predicted_amplification
|
||||||
|
collaborative_analysis,static_mean,82782.45,38030.049999999996,8428.771760000001,4292.238766666668,9.88,6.1066666666666665,1.66,0.9266666666666667,33.798140646040636,17.719989109017618
|
||||||
|
collaborative_analysis,no_context,82782.45,18429.737056529975,8428.771760000001,1933.7549094389146,9.88,4.658333333333333,1.66,0.6333333333333333,33.798140646040636,10.99100144733211
|
||||||
|
collaborative_analysis,full_model,82782.45,65408.811967296744,8428.771760000001,5132.429177599434,9.88,9.033333333333333,1.66,1.4333333333333333,33.798140646040636,25.68477275651994
|
||||||
|
simple_qa,static_mean,5514.23,38030.049999999996,1128.43623,4292.238766666668,3.24,6.1066666666666665,0.11,0.9266666666666667,4.875984197964249,17.719989109017618
|
||||||
|
simple_qa,no_context,5514.23,19556.05971301258,1128.43623,1878.3339705388123,3.24,5.0,0.11,0.65,4.875984197964249,10.442958092513376
|
||||||
|
simple_qa,full_model,5514.23,4564.059252059445,1128.43623,740.9752102854878,3.24,3.2666666666666666,0.11,0.13333333333333333,4.875984197964249,4.07657616682107
|
||||||
|
tool_research,static_mean,25793.47,38030.049999999996,3319.5083100000024,4292.238766666668,5.2,6.1066666666666665,1.01,0.9266666666666667,14.485842483047964,17.719989109017618
|
||||||
|
tool_research,no_context,25793.47,19025.000240049787,3319.5083100000024,1842.727513918263,5.2,4.725,1.01,0.625,14.485842483047964,10.09313023039201
|
||||||
|
tool_research,full_model,25793.47,14558.716131318892,3319.5083100000024,1971.65603294373,5.2,4.775,1.01,0.925,14.485842483047964,10.483430906669906
|
||||||
|
@@ -0,0 +1,6 @@
|
|||||||
|
numpy>=1.20
|
||||||
|
pandas>=1.2
|
||||||
|
matplotlib>=3.3
|
||||||
|
PyYAML>=5.4
|
||||||
|
requests>=2.25
|
||||||
|
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
|
||||||
|
Set-Location $scriptDir
|
||||||
|
|
||||||
|
if (-not $env:OPENAI_API_KEY) {
|
||||||
|
throw '请先设置 OPENAI_API_KEY。也可以直接运行 run_local.ps1 使用本地画像。'
|
||||||
|
}
|
||||||
|
if (-not $env:OPENAI_MODEL) {
|
||||||
|
throw '请先设置 OPENAI_MODEL。'
|
||||||
|
}
|
||||||
|
python run_pipeline.py --generator llm
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
|
||||||
|
Set-Location $scriptDir
|
||||||
|
python run_pipeline.py
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
import argparse
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
|
||||||
|
from src.common import ensure_dirs, load_config, load_json, save_json, seeded_rng
|
||||||
|
from src.data_generator import aggregate_task_truth, generate_event_log, generate_profiles
|
||||||
|
from src.experiments import make_figures, retry_experiment, stress_experiment, validation_experiment
|
||||||
|
from src.parameter_estimation import estimate_parameters
|
||||||
|
|
||||||
|
|
||||||
|
def parse_args():
|
||||||
|
parser = argparse.ArgumentParser(description="Agent network traffic data generation and experiments")
|
||||||
|
parser.add_argument("--config", default="configs/default.yaml")
|
||||||
|
parser.add_argument("--output", default="outputs/latest")
|
||||||
|
parser.add_argument("--generator", choices=["local", "llm"], default="local")
|
||||||
|
parser.add_argument("--steps", default="generate,estimate,experiment")
|
||||||
|
parser.add_argument("--no-fallback", action="store_true")
|
||||||
|
return parser.parse_args()
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
args = parse_args()
|
||||||
|
root = Path(__file__).resolve().parent
|
||||||
|
config_path = (root / args.config).resolve() if not Path(args.config).is_absolute() else Path(args.config)
|
||||||
|
output_path = (root / args.output).resolve() if not Path(args.output).is_absolute() else Path(args.output)
|
||||||
|
cfg = load_config(config_path)
|
||||||
|
dirs = ensure_dirs(output_path)
|
||||||
|
shutil.copy2(config_path, output_path / "config_used.yaml")
|
||||||
|
steps = {x.strip() for x in args.steps.split(",") if x.strip()}
|
||||||
|
rng = seeded_rng(int(cfg["seed"]))
|
||||||
|
|
||||||
|
if "generate" in steps:
|
||||||
|
profiles, metadata = generate_profiles(
|
||||||
|
args.generator, cfg["generation"]["task_types"], cfg["generation"]["llm"],
|
||||||
|
allow_fallback=not args.no_fallback,
|
||||||
|
)
|
||||||
|
save_json({"metadata": metadata, "profiles": profiles}, dirs["data"] / "behavior_profiles.json")
|
||||||
|
train = generate_event_log(profiles, int(cfg["generation"]["train_tasks_per_type"]), rng, cfg["network"])
|
||||||
|
test = generate_event_log(profiles, int(cfg["generation"]["test_tasks_per_type"]), rng, cfg["network"])
|
||||||
|
train.to_csv(dirs["data"] / "train_events.csv", index=False, encoding="utf-8-sig")
|
||||||
|
test.to_csv(dirs["data"] / "test_events.csv", index=False, encoding="utf-8-sig")
|
||||||
|
aggregate_task_truth(test).to_csv(dirs["data"] / "task_truth.csv", index=False, encoding="utf-8-sig")
|
||||||
|
print("[generate] profiles=%d, train_events=%d, test_events=%d" % (len(profiles), len(train), len(test)))
|
||||||
|
|
||||||
|
if "estimate" in steps:
|
||||||
|
train = pd.read_csv(dirs["data"] / "train_events.csv")
|
||||||
|
params = estimate_parameters(train)
|
||||||
|
save_json(params, dirs["parameters"] / "estimated_parameters.json")
|
||||||
|
print("[estimate] task_types=%s" % ",".join(params["task_types"].keys()))
|
||||||
|
|
||||||
|
if "experiment" in steps:
|
||||||
|
params = load_json(dirs["parameters"] / "estimated_parameters.json")
|
||||||
|
truth = pd.read_csv(dirs["data"] / "task_truth.csv")
|
||||||
|
predictions, metrics = validation_experiment(truth, params, cfg, rng)
|
||||||
|
stress = stress_experiment(params, cfg, rng)
|
||||||
|
retry = retry_experiment(params, cfg, rng)
|
||||||
|
predictions.to_csv(dirs["results"] / "validation_predictions.csv", index=False, encoding="utf-8-sig")
|
||||||
|
metrics.to_csv(dirs["results"] / "validation_metrics.csv", index=False, encoding="utf-8-sig")
|
||||||
|
stress.to_csv(dirs["results"] / "stress_results.csv", index=False, encoding="utf-8-sig")
|
||||||
|
retry.to_csv(dirs["results"] / "retry_results.csv", index=False, encoding="utf-8-sig")
|
||||||
|
make_figures(predictions, metrics, stress, retry, dirs["figures"])
|
||||||
|
summary = {
|
||||||
|
"best_model_by_internal_bytes_mape": metrics[metrics["metric"] == "internal_bytes"].sort_values("mape_percent").iloc[0].to_dict(),
|
||||||
|
"best_model_by_latency_mape": metrics[metrics["metric"] == "latency_ms"].sort_values("mape_percent").iloc[0].to_dict(),
|
||||||
|
"max_stress_arrival_rate": float(stress["arrival_rate"].max()),
|
||||||
|
"max_observed_p95_delay_ms": float(stress["p95_delay_ms"].max()),
|
||||||
|
}
|
||||||
|
save_json(summary, dirs["results"] / "summary.json")
|
||||||
|
print("[experiment] results written to %s" % output_path)
|
||||||
|
print(metrics[metrics["metric"].isin(["internal_bytes", "latency_ms"])][["model", "metric", "mape_percent"]].to_string(index=False))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
|
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
"""Agent traffic modeling experiment package."""
|
||||||
|
|
||||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,61 @@
|
|||||||
|
import json
|
||||||
|
import math
|
||||||
|
import os
|
||||||
|
import random
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
|
||||||
|
def load_config(path):
|
||||||
|
with open(path, "r", encoding="utf-8") as f:
|
||||||
|
return yaml.safe_load(f)
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_dirs(root):
|
||||||
|
root = Path(root)
|
||||||
|
dirs = {name: root / name for name in ("data", "parameters", "results", "figures")}
|
||||||
|
for directory in dirs.values():
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
return dirs
|
||||||
|
|
||||||
|
|
||||||
|
def save_json(data, path):
|
||||||
|
with open(path, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(data, f, ensure_ascii=False, indent=2)
|
||||||
|
|
||||||
|
|
||||||
|
def load_json(path):
|
||||||
|
with open(path, "r", encoding="utf-8") as f:
|
||||||
|
return json.load(f)
|
||||||
|
|
||||||
|
|
||||||
|
def seeded_rng(seed):
|
||||||
|
random.seed(seed)
|
||||||
|
np.random.seed(seed)
|
||||||
|
return np.random.default_rng(seed)
|
||||||
|
|
||||||
|
|
||||||
|
def clamp(value, low, high):
|
||||||
|
return max(low, min(high, value))
|
||||||
|
|
||||||
|
|
||||||
|
def lognormal_params_from_mean_cv(mean, cv):
|
||||||
|
mean = max(float(mean), 1e-6)
|
||||||
|
cv = max(float(cv), 1e-6)
|
||||||
|
sigma2 = math.log(cv * cv + 1.0)
|
||||||
|
sigma = math.sqrt(sigma2)
|
||||||
|
mu = math.log(mean) - sigma2 / 2.0
|
||||||
|
return mu, sigma
|
||||||
|
|
||||||
|
|
||||||
|
def sample_lognormal(rng, mean, cv, minimum=1.0):
|
||||||
|
mu, sigma = lognormal_params_from_mean_cv(mean, cv)
|
||||||
|
return max(minimum, float(rng.lognormal(mu, sigma)))
|
||||||
|
|
||||||
|
|
||||||
|
def env(name, default=None):
|
||||||
|
value = os.environ.get(name)
|
||||||
|
return default if value in (None, "") else value
|
||||||
|
|
||||||
@@ -0,0 +1,308 @@
|
|||||||
|
import json
|
||||||
|
import re
|
||||||
|
import uuid
|
||||||
|
from copy import deepcopy
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
import requests
|
||||||
|
|
||||||
|
from .common import clamp, sample_lognormal
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_PROFILES = [
|
||||||
|
{
|
||||||
|
"task_type": "simple_qa",
|
||||||
|
"description": "单节点即可完成的简短问答，少量情况下调用工具。",
|
||||||
|
"phase": "answering",
|
||||||
|
"tool_probability": 0.10,
|
||||||
|
"split_probability": 0.05,
|
||||||
|
"mean_subtasks": 1.2,
|
||||||
|
"timeout_probability": 0.02,
|
||||||
|
"failure_probability": 0.01,
|
||||||
|
"think_time_ms_mean": 420,
|
||||||
|
"think_time_cv": 0.55,
|
||||||
|
"request_size_bytes_mean": 1800,
|
||||||
|
"response_size_bytes_mean": 2600,
|
||||||
|
"message_size_cv": 0.45,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"task_type": "tool_research",
|
||||||
|
"description": "需要搜索、数据库或工具结果的研究任务。",
|
||||||
|
"phase": "evidence_collection",
|
||||||
|
"tool_probability": 0.78,
|
||||||
|
"split_probability": 0.20,
|
||||||
|
"mean_subtasks": 1.8,
|
||||||
|
"timeout_probability": 0.07,
|
||||||
|
"failure_probability": 0.025,
|
||||||
|
"think_time_ms_mean": 900,
|
||||||
|
"think_time_cv": 0.75,
|
||||||
|
"request_size_bytes_mean": 3200,
|
||||||
|
"response_size_bytes_mean": 8500,
|
||||||
|
"message_size_cv": 0.70,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"task_type": "collaborative_analysis",
|
||||||
|
"description": "协调多个执行 Agent 并汇总结果的复杂分析任务。",
|
||||||
|
"phase": "multi_agent_synthesis",
|
||||||
|
"tool_probability": 0.48,
|
||||||
|
"split_probability": 0.82,
|
||||||
|
"mean_subtasks": 3.4,
|
||||||
|
"timeout_probability": 0.09,
|
||||||
|
"failure_probability": 0.035,
|
||||||
|
"think_time_ms_mean": 1450,
|
||||||
|
"think_time_cv": 0.85,
|
||||||
|
"request_size_bytes_mean": 5200,
|
||||||
|
"response_size_bytes_mean": 11800,
|
||||||
|
"message_size_cv": 0.80,
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
PROFILE_KEYS = {
|
||||||
|
"task_type", "description", "phase", "tool_probability", "split_probability",
|
||||||
|
"mean_subtasks", "timeout_probability", "failure_probability",
|
||||||
|
"think_time_ms_mean", "think_time_cv", "request_size_bytes_mean",
|
||||||
|
"response_size_bytes_mean", "message_size_cv",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _extract_json(text):
|
||||||
|
text = text.strip()
|
||||||
|
fenced = re.search(r"```(?:json)?\s*(.*?)```", text, flags=re.S | re.I)
|
||||||
|
if fenced:
|
||||||
|
text = fenced.group(1).strip()
|
||||||
|
start = min([i for i in (text.find("["), text.find("{")) if i >= 0] or [0])
|
||||||
|
text = text[start:]
|
||||||
|
if text.startswith("{"):
|
||||||
|
obj = json.loads(text)
|
||||||
|
return obj.get("profiles", obj)
|
||||||
|
return json.loads(text)
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_profiles(profiles, requested_types):
|
||||||
|
if not isinstance(profiles, list):
|
||||||
|
raise ValueError("LLM output must be a JSON list")
|
||||||
|
validated = []
|
||||||
|
by_type = {p["task_type"]: p for p in DEFAULT_PROFILES}
|
||||||
|
for raw in profiles:
|
||||||
|
if not isinstance(raw, dict) or "task_type" not in raw:
|
||||||
|
continue
|
||||||
|
base = deepcopy(by_type.get(raw["task_type"], DEFAULT_PROFILES[0]))
|
||||||
|
for key in PROFILE_KEYS:
|
||||||
|
if key in raw:
|
||||||
|
base[key] = raw[key]
|
||||||
|
for key in ("tool_probability", "split_probability", "timeout_probability", "failure_probability"):
|
||||||
|
base[key] = clamp(float(base[key]), 0.0, 0.95)
|
||||||
|
base["mean_subtasks"] = clamp(float(base["mean_subtasks"]), 1.0, 8.0)
|
||||||
|
for key in ("think_time_ms_mean", "request_size_bytes_mean", "response_size_bytes_mean"):
|
||||||
|
base[key] = max(float(base[key]), 1.0)
|
||||||
|
for key in ("think_time_cv", "message_size_cv"):
|
||||||
|
base[key] = clamp(float(base[key]), 0.05, 2.5)
|
||||||
|
validated.append(base)
|
||||||
|
found = {p["task_type"] for p in validated}
|
||||||
|
for task_type in requested_types:
|
||||||
|
if task_type not in found:
|
||||||
|
validated.append(deepcopy(by_type.get(task_type, DEFAULT_PROFILES[0])))
|
||||||
|
return [p for p in validated if p["task_type"] in requested_types]
|
||||||
|
|
||||||
|
|
||||||
|
def generate_profiles(mode, task_types, llm_config, allow_fallback=True):
|
||||||
|
if mode == "local":
|
||||||
|
return _validate_profiles(DEFAULT_PROFILES, task_types), {"generator": "local_builtin"}
|
||||||
|
|
||||||
|
import os
|
||||||
|
api_key = os.environ.get("OPENAI_API_KEY")
|
||||||
|
base_url = os.environ.get("OPENAI_BASE_URL", "https://api.openai.com/v1").rstrip("/")
|
||||||
|
model = os.environ.get("OPENAI_MODEL", "")
|
||||||
|
if not api_key or not model:
|
||||||
|
if allow_fallback:
|
||||||
|
return _validate_profiles(DEFAULT_PROFILES, task_types), {
|
||||||
|
"generator": "local_fallback", "reason": "OPENAI_API_KEY or OPENAI_MODEL missing"
|
||||||
|
}
|
||||||
|
raise RuntimeError("LLM mode requires OPENAI_API_KEY and OPENAI_MODEL")
|
||||||
|
|
||||||
|
prompt = f"""你是多智能体网络仿真实验的数据设计专家。请为以下任务类型生成行为画像：
|
||||||
|
{json.dumps(task_types, ensure_ascii=False)}
|
||||||
|
|
||||||
|
仅输出 JSON 数组，每个对象必须包含这些字段：
|
||||||
|
task_type, description, phase, tool_probability, split_probability, mean_subtasks,
|
||||||
|
timeout_probability, failure_probability, think_time_ms_mean, think_time_cv,
|
||||||
|
request_size_bytes_mean, response_size_bytes_mean, message_size_cv。
|
||||||
|
|
||||||
|
约束：概率在0到0.95之间；mean_subtasks在1到8之间；时间单位毫秒；消息大小单位字节；
|
||||||
|
不同任务类型应体现从简单问答、工具密集到多Agent协作的明显差异。画像将用于科学仿真，数值应合理且可解释。"""
|
||||||
|
payload = {
|
||||||
|
"model": model,
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "Return valid JSON only. Do not include markdown commentary."},
|
||||||
|
{"role": "user", "content": prompt},
|
||||||
|
],
|
||||||
|
"temperature": float(llm_config.get("temperature", 0.5)),
|
||||||
|
"max_tokens": int(llm_config.get("max_tokens", 3500)),
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
response = requests.post(
|
||||||
|
base_url + "/chat/completions",
|
||||||
|
headers={"Authorization": "Bearer " + api_key, "Content-Type": "application/json"},
|
||||||
|
json=payload,
|
||||||
|
timeout=float(llm_config.get("timeout_seconds", 90)),
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
body = response.json()
|
||||||
|
content = body["choices"][0]["message"]["content"]
|
||||||
|
profiles = _validate_profiles(_extract_json(content), task_types)
|
||||||
|
return profiles, {"generator": "llm", "model": model, "base_url": base_url}
|
||||||
|
except Exception as exc:
|
||||||
|
if not allow_fallback:
|
||||||
|
raise
|
||||||
|
return _validate_profiles(DEFAULT_PROFILES, task_types), {
|
||||||
|
"generator": "local_fallback", "reason": str(exc), "requested_model": model
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _event(task_id, msg_id, parent_id, timestamp, task_type, phase, source, destination,
|
||||||
|
state_before, state_after, duration, message_type, message_size, queue_length,
|
||||||
|
success=True, retry_count=0, is_external=False):
|
||||||
|
return {
|
||||||
|
"timestamp": round(float(timestamp), 6),
|
||||||
|
"task_id": task_id,
|
||||||
|
"message_id": msg_id,
|
||||||
|
"parent_message_id": parent_id or "",
|
||||||
|
"task_type": task_type,
|
||||||
|
"task_phase": phase,
|
||||||
|
"source": source,
|
||||||
|
"destination": destination,
|
||||||
|
"state_before": state_before,
|
||||||
|
"state_after": state_after,
|
||||||
|
"state_duration_ms": round(float(duration), 3),
|
||||||
|
"message_type": message_type,
|
||||||
|
"message_size_bytes": int(max(0, round(message_size))),
|
||||||
|
"queue_length": int(max(0, queue_length)),
|
||||||
|
"success": bool(success),
|
||||||
|
"retry_count": int(retry_count),
|
||||||
|
"is_external": bool(is_external),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def generate_event_log(profiles, tasks_per_type, rng, network):
|
||||||
|
events = []
|
||||||
|
agent_count = int(network["agent_count"])
|
||||||
|
tool_count = int(network["tool_count"])
|
||||||
|
coordinator = network.get("coordinator", "agent_0")
|
||||||
|
clock = 0.0
|
||||||
|
for profile in profiles:
|
||||||
|
for index in range(tasks_per_type):
|
||||||
|
task_id = "%s_%05d" % (profile["task_type"], index)
|
||||||
|
clock += float(rng.exponential(0.7))
|
||||||
|
t = clock
|
||||||
|
external_size = sample_lognormal(rng, profile["request_size_bytes_mean"] * 0.75,
|
||||||
|
profile["message_size_cv"])
|
||||||
|
root_id = uuid.uuid4().hex[:16]
|
||||||
|
think = sample_lognormal(rng, profile["think_time_ms_mean"], profile["think_time_cv"])
|
||||||
|
queue = int(rng.poisson(0.5 + 2.0 * profile["split_probability"]))
|
||||||
|
events.append(_event(task_id, root_id, "", t, profile["task_type"], profile["phase"],
|
||||||
|
"external", coordinator, "Idle", "Think", think,
|
||||||
|
"external_task", external_size, queue, is_external=True))
|
||||||
|
t += think / 1000.0
|
||||||
|
parent = root_id
|
||||||
|
split = rng.random() < profile["split_probability"]
|
||||||
|
subtask_count = max(1, int(rng.poisson(max(profile["mean_subtasks"] - 1.0, 0.01)) + 1)) if split else 1
|
||||||
|
subtask_count = min(subtask_count, 8)
|
||||||
|
result_sizes = []
|
||||||
|
all_success = True
|
||||||
|
for sub_index in range(subtask_count):
|
||||||
|
worker = "agent_%d" % (1 + ((index + sub_index) % max(1, agent_count - 1)))
|
||||||
|
req_id = uuid.uuid4().hex[:16]
|
||||||
|
req_size = sample_lognormal(rng, profile["request_size_bytes_mean"], profile["message_size_cv"])
|
||||||
|
state = "Split" if split else "CallAgent"
|
||||||
|
events.append(_event(task_id, req_id, parent, t, profile["task_type"], profile["phase"],
|
||||||
|
coordinator, worker, "Think", state, 5.0, "agent_request",
|
||||||
|
req_size, queue))
|
||||||
|
worker_think = sample_lognormal(rng, profile["think_time_ms_mean"], profile["think_time_cv"])
|
||||||
|
t += (8.0 + worker_think) / 1000.0
|
||||||
|
tool_used = rng.random() < profile["tool_probability"]
|
||||||
|
retries = 0
|
||||||
|
success = True
|
||||||
|
tool_response_size = 0.0
|
||||||
|
if tool_used:
|
||||||
|
tool = "tool_%d" % ((index + sub_index) % max(1, tool_count))
|
||||||
|
tool_req_parent = req_id
|
||||||
|
while True:
|
||||||
|
tool_req_id = uuid.uuid4().hex[:16]
|
||||||
|
tool_req_size = sample_lognormal(rng, profile["request_size_bytes_mean"] * 0.35,
|
||||||
|
profile["message_size_cv"])
|
||||||
|
events.append(_event(task_id, tool_req_id, tool_req_parent, t,
|
||||||
|
profile["task_type"], profile["phase"], worker, tool,
|
||||||
|
"Think" if retries == 0 else "Retry", "CallTool", 8.0,
|
||||||
|
"tool_request", tool_req_size, queue, retry_count=retries))
|
||||||
|
t += 0.008
|
||||||
|
timed_out = rng.random() < profile["timeout_probability"]
|
||||||
|
failed = rng.random() < profile["failure_probability"]
|
||||||
|
if not timed_out and not failed:
|
||||||
|
tool_response_size = sample_lognormal(
|
||||||
|
rng, profile["response_size_bytes_mean"] * 0.72,
|
||||||
|
profile["message_size_cv"])
|
||||||
|
response_id = uuid.uuid4().hex[:16]
|
||||||
|
tool_ms = sample_lognormal(rng, profile["think_time_ms_mean"] * 0.65,
|
||||||
|
profile["think_time_cv"])
|
||||||
|
t += tool_ms / 1000.0
|
||||||
|
events.append(_event(task_id, response_id, tool_req_id, t,
|
||||||
|
profile["task_type"], profile["phase"], tool, worker,
|
||||||
|
"CallTool", "Think", tool_ms, "tool_response",
|
||||||
|
tool_response_size, queue, retry_count=retries))
|
||||||
|
break
|
||||||
|
if retries >= 2:
|
||||||
|
success = False
|
||||||
|
fail_id = uuid.uuid4().hex[:16]
|
||||||
|
events.append(_event(task_id, fail_id, tool_req_id, t + 2.2,
|
||||||
|
profile["task_type"], profile["phase"], worker, coordinator,
|
||||||
|
"Wait", "Failed", 2200.0, "error", 600, queue,
|
||||||
|
success=False, retry_count=retries))
|
||||||
|
t += 2.2
|
||||||
|
break
|
||||||
|
retries += 1
|
||||||
|
t += 2.2 + 0.18 * retries
|
||||||
|
result_id = uuid.uuid4().hex[:16]
|
||||||
|
result_size = sample_lognormal(rng, profile["response_size_bytes_mean"],
|
||||||
|
profile["message_size_cv"]) + tool_response_size * 0.15
|
||||||
|
events.append(_event(task_id, result_id, req_id, t, profile["task_type"], profile["phase"],
|
||||||
|
worker, coordinator, "Think", "Send" if success else "Failed", 5.0,
|
||||||
|
"agent_response" if success else "error", result_size if success else 600,
|
||||||
|
queue, success=success, retry_count=retries))
|
||||||
|
result_sizes.append(result_size if success else 600)
|
||||||
|
all_success = all_success and success
|
||||||
|
t += 0.005
|
||||||
|
final_id = uuid.uuid4().hex[:16]
|
||||||
|
final_size = max(800.0, sum(result_sizes) * 0.38)
|
||||||
|
final_think = sample_lognormal(rng, profile["think_time_ms_mean"] * 0.55,
|
||||||
|
profile["think_time_cv"])
|
||||||
|
t += final_think / 1000.0
|
||||||
|
events.append(_event(task_id, final_id, parent, t, profile["task_type"], profile["phase"],
|
||||||
|
coordinator, "external", "Think", "Send" if all_success else "Failed",
|
||||||
|
final_think, "final_response" if all_success else "error", final_size,
|
||||||
|
queue, success=all_success))
|
||||||
|
return pd.DataFrame(events).sort_values(["timestamp", "task_id"]).reset_index(drop=True)
|
||||||
|
|
||||||
|
|
||||||
|
def aggregate_task_truth(events):
|
||||||
|
rows = []
|
||||||
|
for task_id, group in events.groupby("task_id"):
|
||||||
|
group = group.sort_values("timestamp")
|
||||||
|
external = group[group["is_external"]]
|
||||||
|
internal = group[~group["is_external"]]
|
||||||
|
input_bytes = int(external["message_size_bytes"].sum())
|
||||||
|
rows.append({
|
||||||
|
"task_id": task_id,
|
||||||
|
"task_type": group["task_type"].iloc[0],
|
||||||
|
"task_phase": group["task_phase"].iloc[0],
|
||||||
|
"latency_ms": max(0.0, (group["timestamp"].max() - group["timestamp"].min()) * 1000.0),
|
||||||
|
"internal_bytes": int(internal["message_size_bytes"].sum()),
|
||||||
|
"input_bytes": input_bytes,
|
||||||
|
"amplification": float(internal["message_size_bytes"].sum()) / max(input_bytes, 1),
|
||||||
|
"message_count": int(len(internal)),
|
||||||
|
"tool_calls": int((group["message_type"] == "tool_request").sum()),
|
||||||
|
"retries": int(group["retry_count"].max()),
|
||||||
|
"success": bool(group["success"].all()),
|
||||||
|
})
|
||||||
|
return pd.DataFrame(rows)
|
||||||
|
|
||||||
@@ -0,0 +1,152 @@
|
|||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import matplotlib
|
||||||
|
matplotlib.use("Agg")
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import numpy as np
|
||||||
|
import pandas as pd
|
||||||
|
|
||||||
|
from .simulator import simulate_queue_stress, simulate_task_batch
|
||||||
|
|
||||||
|
|
||||||
|
def _metrics(actual, predicted):
|
||||||
|
actual = np.asarray(actual, dtype=float)
|
||||||
|
predicted = np.asarray(predicted, dtype=float)
|
||||||
|
return {
|
||||||
|
"mae": float(np.mean(np.abs(predicted - actual))),
|
||||||
|
"rmse": float(np.sqrt(np.mean((predicted - actual) ** 2))),
|
||||||
|
"mape_percent": float(np.mean(np.abs(predicted - actual) / np.maximum(np.abs(actual), 1e-9)) * 100.0),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def validation_experiment(truth, params, cfg, rng):
|
||||||
|
scenario_truth = truth.groupby("task_type").agg(
|
||||||
|
internal_bytes=("internal_bytes", "mean"),
|
||||||
|
latency_ms=("latency_ms", "mean"),
|
||||||
|
message_count=("message_count", "mean"),
|
||||||
|
tool_calls=("tool_calls", "mean"),
|
||||||
|
amplification=("amplification", "mean"),
|
||||||
|
).reset_index()
|
||||||
|
global_internal = float(scenario_truth["internal_bytes"].mean())
|
||||||
|
global_latency = float(scenario_truth["latency_ms"].mean())
|
||||||
|
rows = []
|
||||||
|
n = int(cfg["experiments"]["validation_tasks_per_scenario"])
|
||||||
|
for _, actual in scenario_truth.iterrows():
|
||||||
|
task_type = actual["task_type"]
|
||||||
|
for model in ("static_mean", "no_context", "full_model"):
|
||||||
|
if model == "static_mean":
|
||||||
|
prediction = {
|
||||||
|
"internal_bytes": global_internal,
|
||||||
|
"latency_ms": global_latency,
|
||||||
|
"message_count": float(scenario_truth["message_count"].mean()),
|
||||||
|
"tool_calls": float(scenario_truth["tool_calls"].mean()),
|
||||||
|
"amplification": float(scenario_truth["amplification"].mean()),
|
||||||
|
}
|
||||||
|
else:
|
||||||
|
samples = pd.DataFrame(simulate_task_batch(
|
||||||
|
task_type, n, params, rng, max_retries=cfg["simulation"]["max_retries"],
|
||||||
|
include_context=(model == "full_model"),
|
||||||
|
))
|
||||||
|
prediction = {key: float(samples[key].mean()) for key in
|
||||||
|
("internal_bytes", "latency_ms", "message_count", "tool_calls", "amplification")}
|
||||||
|
row = {"task_type": task_type, "model": model}
|
||||||
|
for metric, pred in prediction.items():
|
||||||
|
row["actual_" + metric] = float(actual[metric])
|
||||||
|
row["predicted_" + metric] = pred
|
||||||
|
rows.append(row)
|
||||||
|
predictions = pd.DataFrame(rows)
|
||||||
|
metric_rows = []
|
||||||
|
for model, group in predictions.groupby("model"):
|
||||||
|
for metric in ("internal_bytes", "latency_ms", "message_count", "tool_calls", "amplification"):
|
||||||
|
values = _metrics(group["actual_" + metric], group["predicted_" + metric])
|
||||||
|
metric_rows.append({"model": model, "metric": metric, **values})
|
||||||
|
return predictions, pd.DataFrame(metric_rows)
|
||||||
|
|
||||||
|
|
||||||
|
def stress_experiment(params, cfg, rng):
|
||||||
|
task_type = "collaborative_analysis" if "collaborative_analysis" in params["task_types"] else list(params["task_types"])[-1]
|
||||||
|
rows = []
|
||||||
|
for rate in cfg["experiments"]["arrival_rates_per_second"]:
|
||||||
|
rows.append(simulate_queue_stress(
|
||||||
|
float(rate), cfg["experiments"]["stress_duration_seconds"],
|
||||||
|
cfg["experiments"]["stress_warmup_seconds"], task_type, params, rng,
|
||||||
|
max_concurrency=cfg["network"]["max_concurrency"],
|
||||||
|
queue_capacity=cfg["network"]["queue_capacity"],
|
||||||
|
max_retries=cfg["simulation"]["max_retries"],
|
||||||
|
))
|
||||||
|
return pd.DataFrame(rows)
|
||||||
|
|
||||||
|
|
||||||
|
def retry_experiment(params, cfg, rng):
|
||||||
|
task_type = "tool_research" if "tool_research" in params["task_types"] else list(params["task_types"])[0]
|
||||||
|
rows = []
|
||||||
|
for probability in cfg["experiments"]["timeout_probabilities"]:
|
||||||
|
for retry_limit in cfg["experiments"]["retry_limits"]:
|
||||||
|
samples = pd.DataFrame(simulate_task_batch(
|
||||||
|
task_type, 240, params, rng, timeout_probability=float(probability),
|
||||||
|
max_retries=int(retry_limit), include_context=True,
|
||||||
|
))
|
||||||
|
rows.append({
|
||||||
|
"timeout_probability": probability,
|
||||||
|
"max_retries": retry_limit,
|
||||||
|
"success_rate": float(samples["success"].mean()),
|
||||||
|
"mean_retries": float(samples["retries"].mean()),
|
||||||
|
"mean_amplification": float(samples["amplification"].mean()),
|
||||||
|
"mean_latency_ms": float(samples["latency_ms"].mean()),
|
||||||
|
})
|
||||||
|
return pd.DataFrame(rows)
|
||||||
|
|
||||||
|
|
||||||
|
def make_figures(predictions, metrics, stress, retry, figure_dir):
|
||||||
|
figure_dir = Path(figure_dir)
|
||||||
|
plt.rcParams["font.sans-serif"] = ["Microsoft YaHei", "SimHei", "Arial Unicode MS", "DejaVu Sans"]
|
||||||
|
plt.rcParams["axes.unicode_minus"] = False
|
||||||
|
|
||||||
|
full = predictions[predictions["model"] == "full_model"]
|
||||||
|
fig, axes = plt.subplots(1, 2, figsize=(11, 4.5))
|
||||||
|
for ax, metric, label in [(axes[0], "internal_bytes", "内部字节数"), (axes[1], "latency_ms", "任务延迟(ms)")]:
|
||||||
|
x, y = full["actual_" + metric], full["predicted_" + metric]
|
||||||
|
ax.scatter(x, y, s=65, color="#2E74B5")
|
||||||
|
lo, hi = min(x.min(), y.min()), max(x.max(), y.max())
|
||||||
|
ax.plot([lo, hi], [lo, hi], "--", color="#777777")
|
||||||
|
for _, row in full.iterrows():
|
||||||
|
ax.annotate(row["task_type"], (row["actual_" + metric], row["predicted_" + metric]), fontsize=8)
|
||||||
|
ax.set_xlabel("模拟实测值"); ax.set_ylabel("模型预测值"); ax.set_title(label)
|
||||||
|
ax.grid(alpha=0.25)
|
||||||
|
fig.tight_layout(); fig.savefig(figure_dir / "prediction_vs_truth.png", dpi=180); plt.close(fig)
|
||||||
|
|
||||||
|
subset = metrics[metrics["metric"].isin(["internal_bytes", "latency_ms"])]
|
||||||
|
pivot = subset.pivot(index="model", columns="metric", values="mape_percent")
|
||||||
|
pivot = pivot.rename(
|
||||||
|
index={"full_model": "完整模型", "no_context": "无上下文模型", "static_mean": "静态均值模型"},
|
||||||
|
columns={"internal_bytes": "内部字节数", "latency_ms": "任务延迟"},
|
||||||
|
)
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 4.6)); pivot.plot(kind="bar", ax=ax, color=["#2E74B5", "#70AD47"])
|
||||||
|
ax.set_ylabel("MAPE (%)"); ax.set_xlabel("模型"); ax.set_title("完整模型与基线对比"); ax.grid(axis="y", alpha=0.25)
|
||||||
|
ax.legend(title="指标")
|
||||||
|
fig.tight_layout(); fig.savefig(figure_dir / "model_comparison.png", dpi=180); plt.close(fig)
|
||||||
|
|
||||||
|
fig, axes = plt.subplots(2, 2, figsize=(11, 7.5))
|
||||||
|
axes[0, 0].plot(stress["arrival_rate"], stress["throughput_per_second"], marker="o")
|
||||||
|
axes[0, 0].set_title("到达率—吞吐量")
|
||||||
|
axes[0, 1].plot(stress["arrival_rate"], stress["p95_delay_ms"], marker="o", color="#C55A11")
|
||||||
|
axes[0, 1].set_title("到达率—P95延迟")
|
||||||
|
axes[1, 0].plot(stress["arrival_rate"], stress["queue_peak"], marker="o", color="#A5A5A5")
|
||||||
|
axes[1, 0].set_title("到达率—峰值队列")
|
||||||
|
axes[1, 1].plot(stress["arrival_rate"], stress["drop_rate"], marker="o", color="#C00000")
|
||||||
|
axes[1, 1].set_title("到达率—丢弃率")
|
||||||
|
for ax in axes.flat: ax.grid(alpha=0.25); ax.set_xlabel("任务/秒")
|
||||||
|
fig.tight_layout(); fig.savefig(figure_dir / "stress_curves.png", dpi=180); plt.close(fig)
|
||||||
|
|
||||||
|
pivot = retry.pivot(index="timeout_probability", columns="max_retries", values="mean_amplification")
|
||||||
|
fig, ax = plt.subplots(figsize=(8, 5)); image = ax.imshow(pivot.values, aspect="auto", cmap="YlOrRd")
|
||||||
|
ax.set_xticks(range(len(pivot.columns)))
|
||||||
|
ax.set_xticklabels([str(x) for x in pivot.columns])
|
||||||
|
ax.set_yticks(range(len(pivot.index)))
|
||||||
|
ax.set_yticklabels([str(x) for x in pivot.index])
|
||||||
|
ax.set_xlabel("最大重试次数"); ax.set_ylabel("超时概率"); ax.set_title("重试导致的流量放大")
|
||||||
|
for i in range(pivot.shape[0]):
|
||||||
|
for j in range(pivot.shape[1]): ax.text(j, i, "%.2f" % pivot.iloc[i, j], ha="center", va="center", fontsize=8)
|
||||||
|
fig.colorbar(image, ax=ax, label="平均流量放大系数")
|
||||||
|
fig.tight_layout(); fig.savefig(figure_dir / "retry_heatmap.png", dpi=180); plt.close(fig)
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
from collections import defaultdict
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
|
||||||
|
def _stats(series, default=1.0):
|
||||||
|
values = np.asarray(series, dtype=float)
|
||||||
|
values = values[np.isfinite(values)]
|
||||||
|
if len(values) == 0:
|
||||||
|
return {"mean": default, "std": 0.0, "cv": 0.1, "p50": default, "p95": default, "count": 0}
|
||||||
|
mean = float(np.mean(values))
|
||||||
|
std = float(np.std(values))
|
||||||
|
return {
|
||||||
|
"mean": mean,
|
||||||
|
"std": std,
|
||||||
|
"cv": max(0.05, std / max(mean, 1e-9)),
|
||||||
|
"p50": float(np.quantile(values, 0.50)),
|
||||||
|
"p95": float(np.quantile(values, 0.95)),
|
||||||
|
"count": int(len(values)),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def estimate_parameters(events):
|
||||||
|
parameters = {"global": {}, "task_types": {}}
|
||||||
|
transitions = events.groupby(["state_before", "state_after"]).size().to_dict()
|
||||||
|
totals = events.groupby("state_before").size().to_dict()
|
||||||
|
parameters["global"]["transition_probabilities"] = {
|
||||||
|
"%s->%s" % key: float((count + 1.0) / (totals[key[0]] + 8.0))
|
||||||
|
for key, count in transitions.items()
|
||||||
|
}
|
||||||
|
parameters["global"]["duration_ms"] = _stats(events["state_duration_ms"])
|
||||||
|
parameters["global"]["message_size_bytes"] = _stats(events["message_size_bytes"])
|
||||||
|
|
||||||
|
for task_type, group in events.groupby("task_type"):
|
||||||
|
tasks = group.groupby("task_id")
|
||||||
|
task_count = max(1, group["task_id"].nunique())
|
||||||
|
initial_tool_requests = group[(group["message_type"] == "tool_request") & (group["retry_count"] == 0)]
|
||||||
|
agent_requests = group[group["message_type"] == "agent_request"]
|
||||||
|
split_tasks = tasks["state_after"].apply(lambda s: (s == "Split").any())
|
||||||
|
retry_tasks = tasks["retry_count"].max()
|
||||||
|
tool_requests = group[group["message_type"] == "tool_request"]
|
||||||
|
success_tasks = tasks["success"].all()
|
||||||
|
external_sizes = group[group["is_external"]]["message_size_bytes"]
|
||||||
|
request_sizes = group[group["message_type"].isin(["agent_request", "tool_request"])]["message_size_bytes"]
|
||||||
|
response_sizes = group[group["message_type"].isin(["agent_response", "tool_response", "final_response"])]["message_size_bytes"]
|
||||||
|
# 5-8ms values represent message-send bookkeeping, not reasoning/service time.
|
||||||
|
think_durations = group[
|
||||||
|
(group["state_duration_ms"] > 20.0)
|
||||||
|
& (group["message_type"].isin(["external_task", "tool_response", "final_response"]))
|
||||||
|
]["state_duration_ms"]
|
||||||
|
subtask_counts = tasks["message_type"].apply(lambda s: int((s == "agent_request").sum()))
|
||||||
|
parameters["task_types"][task_type] = {
|
||||||
|
"phase": str(group["task_phase"].mode().iloc[0]),
|
||||||
|
"task_count": task_count,
|
||||||
|
"tool_probability": float(len(initial_tool_requests) / max(len(agent_requests), 1)),
|
||||||
|
"split_probability": float(split_tasks.mean()),
|
||||||
|
"mean_subtasks": float(subtask_counts.mean()),
|
||||||
|
"retry_probability": float((tool_requests["retry_count"] > 0).sum() / max(len(tool_requests), 1)),
|
||||||
|
"mean_retries_if_any": float(retry_tasks[retry_tasks > 0].mean()) if (retry_tasks > 0).any() else 0.0,
|
||||||
|
"failure_probability": float((~success_tasks).mean()),
|
||||||
|
"think_time_ms": _stats(think_durations, 500.0),
|
||||||
|
"external_size_bytes": _stats(external_sizes, 1500.0),
|
||||||
|
"request_size_bytes": _stats(request_sizes, 2000.0),
|
||||||
|
"response_size_bytes": _stats(response_sizes, 4000.0),
|
||||||
|
"message_count_per_task": _stats(tasks.size(), 3.0),
|
||||||
|
}
|
||||||
|
return parameters
|
||||||
@@ -0,0 +1,176 @@
|
|||||||
|
import heapq
|
||||||
|
import itertools
|
||||||
|
import math
|
||||||
|
from collections import defaultdict, deque
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
from .common import sample_lognormal
|
||||||
|
|
||||||
|
|
||||||
|
def simulate_task_batch(task_type, task_count, params, rng, timeout_probability=None,
|
||||||
|
max_retries=2, include_context=True):
|
||||||
|
task_params = params["task_types"][task_type] if include_context else _globalized(params)
|
||||||
|
rows = []
|
||||||
|
for _ in range(task_count):
|
||||||
|
split = rng.random() < task_params["split_probability"]
|
||||||
|
subtasks = max(1, int(rng.poisson(max(task_params["mean_subtasks"] - 1.0, 0.01)) + 1)) if split else 1
|
||||||
|
subtasks = min(subtasks, 8)
|
||||||
|
input_size = sample_lognormal(rng, task_params["external_size_bytes"]["mean"],
|
||||||
|
task_params["external_size_bytes"]["cv"])
|
||||||
|
total_bytes = 0.0
|
||||||
|
messages = 0
|
||||||
|
tool_calls = 0
|
||||||
|
retries_total = 0
|
||||||
|
success = True
|
||||||
|
durations = []
|
||||||
|
coordinator_think = sample_lognormal(rng, task_params["think_time_ms"]["mean"],
|
||||||
|
task_params["think_time_ms"]["cv"])
|
||||||
|
durations.append(coordinator_think)
|
||||||
|
for _sub in range(subtasks):
|
||||||
|
tool = rng.random() < task_params["tool_probability"]
|
||||||
|
req = sample_lognormal(rng, task_params["request_size_bytes"]["mean"],
|
||||||
|
task_params["request_size_bytes"]["cv"])
|
||||||
|
total_bytes += req
|
||||||
|
messages += 1
|
||||||
|
worker_time = sample_lognormal(rng, task_params["think_time_ms"]["mean"],
|
||||||
|
task_params["think_time_ms"]["cv"])
|
||||||
|
durations.append(worker_time)
|
||||||
|
if tool:
|
||||||
|
p_timeout = task_params["retry_probability"] if timeout_probability is None else timeout_probability
|
||||||
|
attempts = 0
|
||||||
|
tool_success = False
|
||||||
|
while attempts <= max_retries:
|
||||||
|
tool_calls += 1
|
||||||
|
tool_req = sample_lognormal(rng, task_params["request_size_bytes"]["mean"] * 0.35,
|
||||||
|
task_params["request_size_bytes"]["cv"])
|
||||||
|
total_bytes += tool_req
|
||||||
|
messages += 1
|
||||||
|
timed_out = rng.random() < p_timeout
|
||||||
|
if not timed_out:
|
||||||
|
tool_resp = sample_lognormal(rng, task_params["response_size_bytes"]["mean"] * 0.72,
|
||||||
|
task_params["response_size_bytes"]["cv"])
|
||||||
|
total_bytes += tool_resp
|
||||||
|
messages += 1
|
||||||
|
durations.append(sample_lognormal(rng, task_params["think_time_ms"]["mean"] * 0.65,
|
||||||
|
task_params["think_time_ms"]["cv"]))
|
||||||
|
tool_success = True
|
||||||
|
break
|
||||||
|
if attempts < max_retries:
|
||||||
|
retries_total += 1
|
||||||
|
durations.append(2200.0 + 180.0 * (attempts + 1))
|
||||||
|
attempts += 1
|
||||||
|
success = success and tool_success
|
||||||
|
response = sample_lognormal(rng, task_params["response_size_bytes"]["mean"],
|
||||||
|
task_params["response_size_bytes"]["cv"])
|
||||||
|
total_bytes += response if success else 600.0
|
||||||
|
messages += 1
|
||||||
|
final_size = max(800.0, subtasks * task_params["response_size_bytes"]["mean"] * 0.38)
|
||||||
|
total_bytes += final_size
|
||||||
|
messages += 1
|
||||||
|
latency = sum(durations) + 8.0 * messages
|
||||||
|
rows.append({
|
||||||
|
"task_type": task_type,
|
||||||
|
"internal_bytes": total_bytes,
|
||||||
|
"input_bytes": input_size,
|
||||||
|
"amplification": total_bytes / max(input_size, 1.0),
|
||||||
|
"latency_ms": latency,
|
||||||
|
"message_count": messages,
|
||||||
|
"tool_calls": tool_calls,
|
||||||
|
"retries": retries_total,
|
||||||
|
"success": success,
|
||||||
|
})
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def _globalized(params):
|
||||||
|
entries = list(params["task_types"].values())
|
||||||
|
def mean(name):
|
||||||
|
return float(np.mean([e[name] for e in entries]))
|
||||||
|
def stat(name):
|
||||||
|
return {
|
||||||
|
"mean": float(np.mean([e[name]["mean"] for e in entries])),
|
||||||
|
"cv": float(np.mean([e[name]["cv"] for e in entries])),
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"tool_probability": mean("tool_probability"),
|
||||||
|
"split_probability": mean("split_probability"),
|
||||||
|
"mean_subtasks": mean("mean_subtasks"),
|
||||||
|
"retry_probability": mean("retry_probability"),
|
||||||
|
"failure_probability": mean("failure_probability"),
|
||||||
|
"think_time_ms": stat("think_time_ms"),
|
||||||
|
"external_size_bytes": stat("external_size_bytes"),
|
||||||
|
"request_size_bytes": stat("request_size_bytes"),
|
||||||
|
"response_size_bytes": stat("response_size_bytes"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def simulate_queue_stress(arrival_rate, duration_seconds, warmup_seconds, task_type, params,
|
||||||
|
rng, max_concurrency=4, queue_capacity=500, timeout_probability=None,
|
||||||
|
max_retries=2):
|
||||||
|
tasks = simulate_task_batch(
|
||||||
|
task_type, max(200, int(arrival_rate * duration_seconds * 1.5)), params, rng,
|
||||||
|
timeout_probability=timeout_probability, max_retries=max_retries,
|
||||||
|
)
|
||||||
|
event_queue = []
|
||||||
|
counter = itertools.count()
|
||||||
|
time = 0.0
|
||||||
|
task_index = 0
|
||||||
|
while time < duration_seconds:
|
||||||
|
time += float(rng.exponential(1.0 / max(arrival_rate, 1e-9)))
|
||||||
|
if time <= duration_seconds:
|
||||||
|
heapq.heappush(event_queue, (time, next(counter), "arrival", task_index))
|
||||||
|
task_index += 1
|
||||||
|
|
||||||
|
waiting = deque()
|
||||||
|
busy = 0
|
||||||
|
completed = []
|
||||||
|
dropped = 0
|
||||||
|
queue_peak = 0
|
||||||
|
arrival_times = {}
|
||||||
|
while event_queue:
|
||||||
|
now, _, event_type, idx = heapq.heappop(event_queue)
|
||||||
|
if event_type == "arrival":
|
||||||
|
arrival_times[idx] = now
|
||||||
|
if busy < max_concurrency:
|
||||||
|
busy += 1
|
||||||
|
task = tasks[idx % len(tasks)]
|
||||||
|
service = task["latency_ms"] / 1000.0
|
||||||
|
heapq.heappush(event_queue, (now + service, next(counter), "complete", idx))
|
||||||
|
elif len(waiting) < queue_capacity:
|
||||||
|
waiting.append(idx)
|
||||||
|
queue_peak = max(queue_peak, len(waiting))
|
||||||
|
else:
|
||||||
|
dropped += 1
|
||||||
|
else:
|
||||||
|
task = tasks[idx % len(tasks)]
|
||||||
|
if arrival_times[idx] >= warmup_seconds:
|
||||||
|
record = dict(task)
|
||||||
|
record["end_to_end_ms"] = (now - arrival_times[idx]) * 1000.0
|
||||||
|
completed.append(record)
|
||||||
|
busy -= 1
|
||||||
|
if waiting:
|
||||||
|
nxt = waiting.popleft()
|
||||||
|
busy += 1
|
||||||
|
task2 = tasks[nxt % len(tasks)]
|
||||||
|
heapq.heappush(event_queue, (now + task2["latency_ms"] / 1000.0,
|
||||||
|
next(counter), "complete", nxt))
|
||||||
|
offered = max(1, task_index)
|
||||||
|
if completed:
|
||||||
|
delays = np.array([x["end_to_end_ms"] for x in completed])
|
||||||
|
amps = np.array([x["amplification"] for x in completed])
|
||||||
|
success = np.array([x["success"] for x in completed], dtype=float)
|
||||||
|
else:
|
||||||
|
delays, amps, success = np.array([0.0]), np.array([0.0]), np.array([0.0])
|
||||||
|
return {
|
||||||
|
"arrival_rate": arrival_rate,
|
||||||
|
"offered_tasks": offered,
|
||||||
|
"completed_tasks": len(completed),
|
||||||
|
"throughput_per_second": len(completed) / max(duration_seconds - warmup_seconds, 1),
|
||||||
|
"mean_delay_ms": float(delays.mean()),
|
||||||
|
"p95_delay_ms": float(np.quantile(delays, 0.95)),
|
||||||
|
"queue_peak": queue_peak,
|
||||||
|
"drop_rate": dropped / offered,
|
||||||
|
"success_rate": float(success.mean()),
|
||||||
|
"mean_amplification": float(amps.mean()),
|
||||||
|
}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
param([Parameter(Mandatory=$true)][string]$OutputPath)
|
||||||
|
$ErrorActionPreference='Stop'
|
||||||
|
$word=New-Object -ComObject Word.Application
|
||||||
|
$word.Visible=$false
|
||||||
|
$word.DisplayAlerts=0
|
||||||
|
try {
|
||||||
|
$doc=$word.Documents.Add()
|
||||||
|
try {
|
||||||
|
$doc.PageSetup.PaperSize=2
|
||||||
|
$doc.PageSetup.TopMargin=$word.InchesToPoints(0.85)
|
||||||
|
$doc.PageSetup.BottomMargin=$word.InchesToPoints(0.8)
|
||||||
|
$doc.PageSetup.LeftMargin=$word.InchesToPoints(0.9)
|
||||||
|
$doc.PageSetup.RightMargin=$word.InchesToPoints(0.9)
|
||||||
|
$normal=$doc.Styles.Item(-1)
|
||||||
|
$normal.Font.Name='Calibri'; $normal.Font.NameFarEast='Microsoft YaHei'; $normal.Font.Size=10.5
|
||||||
|
$normal.ParagraphFormat.SpaceAfter=6; $normal.ParagraphFormat.LineSpacingRule=5; $normal.ParagraphFormat.LineSpacing=15
|
||||||
|
$title=$doc.Styles.Item(-63)
|
||||||
|
$title.Font.Name='Calibri'; $title.Font.NameFarEast='Microsoft YaHei'; $title.Font.Size=25; $title.Font.Bold=$true; $title.Font.Color=9655585
|
||||||
|
$title.ParagraphFormat.Alignment=1; $title.ParagraphFormat.SpaceBefore=100; $title.ParagraphFormat.SpaceAfter=12
|
||||||
|
$subtitle=$doc.Styles.Item(-75)
|
||||||
|
$subtitle.Font.Name='Calibri'; $subtitle.Font.NameFarEast='Microsoft YaHei'; $subtitle.Font.Size=14; $subtitle.Font.Color=8421504
|
||||||
|
$subtitle.ParagraphFormat.Alignment=1; $subtitle.ParagraphFormat.SpaceAfter=24
|
||||||
|
foreach($pair in @(@(-2,16,16,8,11621185),@(-3,13,12,6,11621185),@(-4,11.5,9,4,9655585))){
|
||||||
|
$s=$doc.Styles.Item($pair[0]); $s.Font.Name='Calibri'; $s.Font.NameFarEast='Microsoft YaHei'; $s.Font.Size=$pair[1]; $s.Font.Bold=$true; $s.Font.Color=$pair[4]
|
||||||
|
$s.ParagraphFormat.SpaceBefore=$pair[2]; $s.ParagraphFormat.SpaceAfter=$pair[3]; $s.ParagraphFormat.KeepWithNext=$true
|
||||||
|
}
|
||||||
|
$doc.SaveAs2($OutputPath,16)
|
||||||
|
} finally { $doc.Close($true); [Runtime.InteropServices.Marshal]::ReleaseComObject($doc)|Out-Null }
|
||||||
|
} finally { $word.Quit(); [Runtime.InteropServices.Marshal]::ReleaseComObject($word)|Out-Null }
|
||||||
@@ -0,0 +1,357 @@
|
|||||||
|
---
|
||||||
|
title: "大规模多智能体网络流量建模与预测"
|
||||||
|
subtitle: "整体设计思路（参赛材料初稿）"
|
||||||
|
author: "参赛团队：待填写"
|
||||||
|
date: "2026年8月"
|
||||||
|
toc: true
|
||||||
|
toc-title: "目录"
|
||||||
|
number-sections: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# 本次修订说明
|
||||||
|
|
||||||
|
**修订日期：2026年8月13日**
|
||||||
|
|
||||||
|
本版本在原有整体设计框架上增加了“模拟实验设计流程”，主要改动如下：
|
||||||
|
|
||||||
|
- 在“数据与参数设计”章节新增 **5.3 模拟实验设计流程**；
|
||||||
|
- 增加“场景定义—大模型生成行为画像—参数校验—结构化日志生成—训练集估参—验证集调参—测试集评估—实验输出”的完整闭环；
|
||||||
|
- 说明大模型只生成少量任务行为画像，本地程序负责扩展大量事件日志，以保证消息关系、时间顺序和统计口径一致；
|
||||||
|
- 增加模拟实验各环节的工作内容和质量控制表；
|
||||||
|
- 明确实验支持无 API Key 的本地画像模式和兼容 OpenAI 接口的 LLM 模式；
|
||||||
|
- 将答辩 PPT 建议由 14 页调整为 15 页，新增“模拟实验设计流程”展示页；
|
||||||
|
- 强调模拟数据用于快速验证模型机制，不等同于真实生产日志，后续需要使用真实数据校准。
|
||||||
|
|
||||||
|
对应实验代码位于 `agent_traffic_experiments/`，可生成事件日志、估计参数并运行预测验证、基线对比、压力和重试实验。
|
||||||
|
|
||||||
|
# 项目摘要
|
||||||
|
|
||||||
|
随着大语言模型从单体问答逐步发展为多智能体协作系统，任务拆分、Agent 间协商、工具调用、数据查询、结果回传和超时重试会产生大量内部消息。系统规模扩大后，通信流量不再只由用户请求量决定，还受到节点能力、任务阶段、网络拓扑、队列负载和故障重试的共同影响。传统的静态流量估算或单一排队模型难以还原这种“业务行为驱动网络流量”的动态过程。
|
||||||
|
|
||||||
|
本项目提出一套面向大规模 Agent 网络的流量建模与预测方案。方案以动态通信图描述节点连接，以外层状态机描述消息跨节点传播，以内层状态机描述节点接收、排队、思考、调用、等待、重试和发送等行为，并采用离散事件仿真计算节点流量、链路负载、端到端时延、队列长度和故障放大效应。模型参数可从小规模 Agent 运行日志和压力测试中估计，再通过分层参数和能力特征推广到更大规模网络。
|
||||||
|
|
||||||
|
> 核心设计：动态通信图 + 双层随机状态机 + 消息队列 + 离散事件仿真 + 分层参数学习 + 流量与拥塞评估。
|
||||||
|
|
||||||
|
# 一、研究背景与问题提出
|
||||||
|
|
||||||
|
## 1.1 背景
|
||||||
|
|
||||||
|
典型的多智能体系统通常包含协调 Agent、执行 Agent、检索服务、工具服务、数据库以及模型推理服务。一次外部任务可能经历如下过程：
|
||||||
|
|
||||||
|
1. 协调节点解析任务并拆分子任务；
|
||||||
|
2. 多个执行节点并行处理；
|
||||||
|
3. 执行节点调用工具或查询数据；
|
||||||
|
4. 工具结果返回后继续推理；
|
||||||
|
5. 失败请求触发超时、重试或改道；
|
||||||
|
6. 协调节点汇总结果并返回用户。
|
||||||
|
|
||||||
|
因此，一条外部请求可能放大为数十条内部消息。随着 Agent 数量和并发任务增加，系统会出现热点节点、队列堆积、链路拥塞、超时重试放大等现象。若无法提前预测这些行为，就难以完成容量规划、资源调度和可靠性设计。
|
||||||
|
|
||||||
|
## 1.2 现有方法的不足
|
||||||
|
|
||||||
|
- **静态倍数估算**只能给出平均流量，不能描述任务阶段和故障状态下的变化。
|
||||||
|
- **单纯拓扑模型**只能说明节点是否可达，不能说明节点为何产生新消息。
|
||||||
|
- **单一排队模型**能够分析等待时间，但难以表达拆分、工具调用和多轮协作。
|
||||||
|
- **完全数据驱动模型**需要大量大规模真实日志，冷启动成本高且解释性不足。
|
||||||
|
|
||||||
|
## 1.3 核心问题
|
||||||
|
|
||||||
|
本项目试图解决三个问题：
|
||||||
|
|
||||||
|
1. 如何用统一模型描述不同 Agent、工具和服务的通信行为？
|
||||||
|
2. 如何从小规模日志估计参数，并推演大规模网络的流量和延迟？
|
||||||
|
3. 如何刻画“拥塞—超时—重试—流量增加”的动态反馈过程？
|
||||||
|
|
||||||
|
# 二、项目目标与应用价值
|
||||||
|
|
||||||
|
## 2.1 建模目标
|
||||||
|
|
||||||
|
给定网络拓扑、节点能力、外部任务负载及行为参数，模型输出：
|
||||||
|
|
||||||
|
- 每个节点的入站和出站消息率、字节流量；
|
||||||
|
- 每条链路的消息数、字节数及带宽利用率；
|
||||||
|
- 任务平均时延以及 P95、P99 尾延迟；
|
||||||
|
- 节点队列长度、资源利用率和吞吐量；
|
||||||
|
- 消息平均跳数、任务内部消息数和流量放大系数；
|
||||||
|
- 超时、失败和重试条件下的额外流量；
|
||||||
|
- 热点节点、瓶颈链路和系统容量拐点。
|
||||||
|
|
||||||
|
## 2.2 应用价值
|
||||||
|
|
||||||
|
| 应用场景 | 模型提供的能力 | 可支持的决策 |
|
||||||
|
|---|---|---|
|
||||||
|
| 上线前容量规划 | 预测不同任务量下的节点与链路负载 | 配置并发数、实例数和带宽 |
|
||||||
|
| Agent 编排优化 | 比较不同拓扑和路由策略 | 减少跳数、热点与无效协作 |
|
||||||
|
| 故障与可靠性设计 | 模拟节点故障、超时和重试 | 选择重试次数、退避策略和备用节点 |
|
||||||
|
| 在线运维 | 根据日志校准模型并识别偏差 | 发现异常流量和潜在拥塞 |
|
||||||
|
| 成本评估 | 估计内部消息量、字节量和工具调用次数 | 对比不同架构的通信与计算成本 |
|
||||||
|
|
||||||
|
# 三、总体设计框架
|
||||||
|
|
||||||
|
## 3.1 总体技术路线
|
||||||
|
|
||||||
|
项目分为数据层、参数层、模型层、仿真层和评估层。
|
||||||
|
|
||||||
|
```text
|
||||||
|
Agent运行日志、拓扑配置、压力测试数据
|
||||||
|
↓
|
||||||
|
数据清洗、任务链还原、特征提取
|
||||||
|
↓
|
||||||
|
状态转移概率 / 持续时间 / 消息大小 / 失败与重试参数
|
||||||
|
↓
|
||||||
|
动态通信图 + 外层传播状态机 + 内层节点状态机 + 队列模型
|
||||||
|
↓
|
||||||
|
离散事件仿真
|
||||||
|
↓
|
||||||
|
节点流量 / 链路流量 / 延迟 / 吞吐量 / 拥塞 / 可靠性
|
||||||
|
↓
|
||||||
|
实测对比、基线对比、消融和压力实验
|
||||||
|
```
|
||||||
|
|
||||||
|
## 3.2 五层架构
|
||||||
|
|
||||||
|
| 层次 | 主要内容 | 关键产物 |
|
||||||
|
|---|---|---|
|
||||||
|
| 数据层 | 采集任务、消息、状态、队列和链路日志 | 标准化事件日志 |
|
||||||
|
| 参数层 | 估计概率、分布、容量和路由参数 | 参数配置与置信度 |
|
||||||
|
| 模型层 | 图模型、双层状态机、队列和流量公式 | 数学模型 |
|
||||||
|
| 仿真层 | 事件队列、状态更新、消息生成和统计 | 可运行仿真器 |
|
||||||
|
| 评估层 | 正确性、精度、扩展性和鲁棒性验证 | 实验报告与图表 |
|
||||||
|
|
||||||
|
# 四、核心模型设计
|
||||||
|
|
||||||
|
## 4.1 动态通信图
|
||||||
|
|
||||||
|
系统表示为随时间变化的有向图：
|
||||||
|
|
||||||
|
$$
|
||||||
|
G(t)=(V(t),E(t))
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中节点可以是 Agent、模型服务、工具或数据库，边表示允许的通信关系。节点不被永久划分为固定角色，而是通过能力向量描述：
|
||||||
|
|
||||||
|
```text
|
||||||
|
can_reason, can_split_task, can_call_tool,
|
||||||
|
can_query_database, can_forward, max_concurrency
|
||||||
|
```
|
||||||
|
|
||||||
|
同一个节点可以在不同任务阶段承担协调、执行、查询或转发功能，从而提高模型对异构系统和新节点的适应能力。
|
||||||
|
|
||||||
|
## 4.2 外层状态机：消息如何跨节点传播
|
||||||
|
|
||||||
|
外层状态机描述消息当前所在节点、路径位置、交互上下文以及下一目标节点。它回答“消息去哪里”的问题。
|
||||||
|
|
||||||
|
```text
|
||||||
|
外部任务 → 协调节点 → 执行节点 → 工具节点
|
||||||
|
↓ ↑
|
||||||
|
等待结果 ← 返回结果
|
||||||
|
↓
|
||||||
|
协调节点 → 外部系统
|
||||||
|
```
|
||||||
|
|
||||||
|
目标节点选择同时考虑可达性、节点能力、网络距离、队列负载和链路利用率。
|
||||||
|
|
||||||
|
## 4.3 内层状态机：节点收到消息后做什么
|
||||||
|
|
||||||
|
模型严格区分节点资源状态和消息处理状态：节点资源状态 $q_v\in\{Idle,Busy,Saturated,Failed\}$ 描述节点整体的资源占用情况；消息处理状态 $q_m$ 描述一条消息或任务处理上下文当前所处的业务阶段。下面的状态机属于 $q_m$，同一节点可以同时维护多个 $q_m$，但活跃上下文数量不得超过 `max_concurrency`。
|
||||||
|
|
||||||
|
所有节点共享一套通用状态集合：
|
||||||
|
|
||||||
|
```text
|
||||||
|
Idle → Receive → Queue → Think
|
||||||
|
├→ Send
|
||||||
|
├→ Split → Wait
|
||||||
|
├→ CallAgent → Wait
|
||||||
|
├→ CallTool → Wait
|
||||||
|
├→ Query → Wait
|
||||||
|
└→ Forward → Wait
|
||||||
|
Wait → Think / Retry → Failed
|
||||||
|
Send / Failed → Idle
|
||||||
|
```
|
||||||
|
|
||||||
|
能力和拓扑先决定某个动作是否可行，条件概率模型再在可行动作中选择下一状态。例如，没有工具调用能力或邻域中没有可用工具时，`Think → CallTool` 的概率直接为零。
|
||||||
|
|
||||||
|
## 4.4 分层参数与小样本泛化
|
||||||
|
|
||||||
|
状态转移概率不是所有节点共用的固定常数，而是由任务、阶段、能力、拓扑和负载共同决定：
|
||||||
|
|
||||||
|
$$
|
||||||
|
P(q'\mid q,c,x_v,z_v,m,\ell_v)
|
||||||
|
$$
|
||||||
|
|
||||||
|
采用“全局基础参数—任务修正—阶段修正—能力修正—拓扑与负载修正”的分层结构。新任务或新节点数据不足时，先回退到相似类别或全局参数，并记录参数来源和可信度；获得新日志后再增量校准。
|
||||||
|
|
||||||
|
## 4.5 消息队列与拥塞反馈
|
||||||
|
|
||||||
|
节点队列随到达和服务动态变化：
|
||||||
|
|
||||||
|
$$
|
||||||
|
Q_v(t+\Delta t)=\max\{0,Q_v(t)+A_v(t)-D_v(t)\}
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中 $Q_v(t)$ 只统计已经到达节点、但尚未获得活跃处理资源的消息；正在 `Think`、`CallTool` 或 `Wait` 中的上下文不计入队列。$D_v$ 表示从等待队列取出并开始处理的消息数，而不是处理完成数。队列采用有界、按 `priority` 优先且同优先级按到达顺序处理的规则。默认情况下 `Wait` 状态仍占用并发槽；若实际系统等待期间释放资源，应显式配置 `wait_holds_slot=false`。
|
||||||
|
|
||||||
|
当到达率接近或超过处理能力时，等待时间增加并可能触发超时。超时产生重试消息，进一步增加队列和链路负载，从而形成反馈闭环：
|
||||||
|
|
||||||
|
```text
|
||||||
|
负载上升 → 排队增长 → 延迟上升 → 超时增多
|
||||||
|
↑ ↓
|
||||||
|
└──────── 重试消息增加 ←───────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4.6 离散事件仿真
|
||||||
|
|
||||||
|
采用事件驱动方式，只在事件发生时更新状态。主要事件包括外部任务到达、消息到达、处理开始、状态完成、发送完成、工具返回、请求超时、重试和故障。
|
||||||
|
|
||||||
|
每次处理事件时：
|
||||||
|
|
||||||
|
1. 读取节点、消息、任务阶段和当前负载；
|
||||||
|
2. 过滤不可行的状态转移；
|
||||||
|
3. 计算并抽样下一状态及持续时间；
|
||||||
|
4. 生成输出消息并选择目标节点；
|
||||||
|
5. 更新节点队列、链路状态和流量统计；
|
||||||
|
6. 把后续事件加入全局优先队列。
|
||||||
|
|
||||||
|
# 五、数据与参数设计
|
||||||
|
|
||||||
|
## 5.1 最小日志字段
|
||||||
|
|
||||||
|
| 字段组 | 字段 | 主要用途 |
|
||||||
|
|---|---|---|
|
||||||
|
| 标识 | task_id、message_id、parent_message_id、correlation_id、attempt_id | 还原任务调用链、匹配请求响应并区分重试尝试 |
|
||||||
|
| 时间 | timestamp、state_start、state_end | 估计状态时间和端到端延迟 |
|
||||||
|
| 路由 | source、destination、next_hop、path、hop | 区分逻辑终点与实际下一跳，统计有向链路流量 |
|
||||||
|
| 业务 | task_type、task_phase、message_type | 分层估计参数 |
|
||||||
|
| 状态 | state_before、state_after | 估计状态转移概率 |
|
||||||
|
| 负载 | queue_length、concurrency、link_utilization | 估计排队和拥塞效应 |
|
||||||
|
| 结果 | message_size、success、retry_count | 估计字节流量与可靠性 |
|
||||||
|
|
||||||
|
## 5.2 参数来源
|
||||||
|
|
||||||
|
- 系统配置：节点能力、并发上限、队列容量和链路带宽；
|
||||||
|
- 小规模日志：状态转移、消息大小、目标选择和处理时间；
|
||||||
|
- 压力测试：高负载下服务速度、超时率和失败率；
|
||||||
|
- 场景假设：尚无数据参数的初始范围；
|
||||||
|
- 仿真校准：根据验证集误差选择分布和调整参数；测试集仅用于最终评估。
|
||||||
|
|
||||||
|
## 5.3 模拟实验设计流程
|
||||||
|
|
||||||
|
在真实大规模 Agent 日志暂不充足的阶段，项目采用“大模型生成行为画像、本地程序扩展结构化日志、状态机模型完成仿真验证”的快速实验路线。大模型不直接逐行编造数万条日志，而是生成少量可解释的任务行为参数，再由受约束的数据生成器保证消息关系、时间顺序和流量统计一致。
|
||||||
|
|
||||||
|
```text
|
||||||
|
设定任务类型与网络场景
|
||||||
|
↓
|
||||||
|
大模型生成任务行为画像
|
||||||
|
工具调用率 / 拆分率 / 子任务数 / 处理时间 / 消息大小 / 超时率
|
||||||
|
↓
|
||||||
|
画像校验与参数约束
|
||||||
|
概率范围检查 / 必填字段检查 / 缺失参数回退
|
||||||
|
↓
|
||||||
|
本地随机引擎生成结构化事件日志
|
||||||
|
task_id / message_id / 状态转移 / 节点路径 / 字节数 / 时间戳
|
||||||
|
↓
|
||||||
|
训练集估计模型参数，验证集选择分布和调整参数
|
||||||
|
↓
|
||||||
|
测试集仅保留为模拟实测值并用于最终评估
|
||||||
|
↓
|
||||||
|
运行双层状态机与离散事件仿真
|
||||||
|
↓
|
||||||
|
基线对比 / 压力实验 / 重试放大实验
|
||||||
|
↓
|
||||||
|
输出 CSV、参数 JSON 和报告图表
|
||||||
|
```
|
||||||
|
|
||||||
|
| 环节 | 主要工作 | 质量控制 |
|
||||||
|
|---|---|---|
|
||||||
|
| 场景定义 | 设置简单问答、工具研究、多 Agent 协作等任务 | 保证任务复杂度具有明显梯度 |
|
||||||
|
| LLM 画像生成 | 生成状态概率、处理时间、消息大小和故障参数 | 限定字段、单位和数值范围 |
|
||||||
|
| 日志扩展 | 将画像扩展为可统计的任务与消息事件 | 保证 ID 唯一、父子消息可追踪、时间单调 |
|
||||||
|
| 参数估计 | 从训练日志估计分层模型参数 | 测试数据不参与估参 |
|
||||||
|
| 模型实验 | 执行预测、基线、压力和重试实验 | 固定随机种子，多次重复 |
|
||||||
|
| 结果解释 | 形成误差表、容量曲线和重试热力图 | 明确标注为模拟数据，不替代真实验证 |
|
||||||
|
|
||||||
|
当前实验工程支持两种模式：没有 API Key 时使用内置行为画像立即跑通；配置兼容 OpenAI 接口后，由大模型生成画像。后续获得真实日志时，可以保持实验流程不变，只替换或校准画像和参数。
|
||||||
|
|
||||||
|
# 六、方案创新点
|
||||||
|
|
||||||
|
## 6.1 双层状态机实现业务行为与网络传播解耦
|
||||||
|
|
||||||
|
外层描述消息路径，内层描述节点行为。两层分别清晰、组合后又能完整计算流量，避免把所有节点和动作塞入一个不可维护的巨型状态机。
|
||||||
|
|
||||||
|
## 6.2 能力驱动的统一节点表示
|
||||||
|
|
||||||
|
使用能力向量和上下文决定行为，不依赖固定节点类型，使模型能够支持新节点、角色动态变化和异构 Agent 网络。
|
||||||
|
|
||||||
|
## 6.3 小规模日志到大规模网络的分层推广
|
||||||
|
|
||||||
|
通过共享基础参数、条件修正和回退机制减少对海量真实数据的依赖，同时保留参数解释性和可信度标记。
|
||||||
|
|
||||||
|
## 6.4 显式描述重试流量放大闭环
|
||||||
|
|
||||||
|
模型不仅计算正常通信，还刻画排队、超时、失败、重试对流量的反向影响，可用于发现系统容量拐点和故障雪崩风险。
|
||||||
|
|
||||||
|
# 七、验证思路
|
||||||
|
|
||||||
|
验证工作分为五个层次：
|
||||||
|
|
||||||
|
1. **程序正确性**：三节点手工算例与仿真逐事件对账；
|
||||||
|
2. **预测准确性**：训练日志估参、测试日志比较节点流量和延迟；
|
||||||
|
3. **模型必要性**：与静态倍数、纯拓扑和简单排队基线比较；
|
||||||
|
4. **模块贡献**：移除队列、阶段、拓扑或重试模块进行消融；
|
||||||
|
5. **大规模能力**：在 10 至 100000 个仿真节点上测试时间和内存开销。
|
||||||
|
|
||||||
|
主要指标包括 MAE、RMSE、MAPE、P95/P99 延迟误差、热点识别准确率、事件处理吞吐量和内存占用。
|
||||||
|
|
||||||
|
# 八、预期成果
|
||||||
|
|
||||||
|
## 8.1 软件成果
|
||||||
|
|
||||||
|
- 网络场景与参数配置模块；
|
||||||
|
- 日志解析和参数估计模块；
|
||||||
|
- 双层状态机离散事件仿真器；
|
||||||
|
- 节点、链路和任务级统计模块；
|
||||||
|
- 实验脚本和可视化看板。
|
||||||
|
|
||||||
|
## 8.2 文档成果
|
||||||
|
|
||||||
|
- 整体设计思路；
|
||||||
|
- 数学建模方案；
|
||||||
|
- 验证与评估报告；
|
||||||
|
- 用户说明、参数字典和复现实验说明。
|
||||||
|
|
||||||
|
## 8.3 预期图表
|
||||||
|
|
||||||
|
- 总体技术路线图和双层状态机图；
|
||||||
|
- 网络节点负载热力图与链路流量图；
|
||||||
|
- 预测值与实测值对比图；
|
||||||
|
- 到达率—吞吐量—延迟曲线；
|
||||||
|
- 超时概率—重试流量放大曲线；
|
||||||
|
- 节点规模—仿真运行时间/内存曲线。
|
||||||
|
|
||||||
|
# 九、实施计划与风险控制
|
||||||
|
|
||||||
|
| 阶段 | 工作内容 | 阶段产物 |
|
||||||
|
|---|---|---|
|
||||||
|
| 第一阶段 | 固化问题、状态和参数定义 | 建模方案 V1 |
|
||||||
|
| 第二阶段 | 实现 3—10 节点最小仿真 | 可运行原型与手工对账 |
|
||||||
|
| 第三阶段 | 构造日志、完成参数估计和基线 | 数据集与实验脚本 |
|
||||||
|
| 第四阶段 | 压力、消融、故障和规模实验 | 实验结果与图表 |
|
||||||
|
| 第五阶段 | 完成报告、PPT和答辩材料 | 正式参赛材料 |
|
||||||
|
|
||||||
|
主要风险及应对措施：
|
||||||
|
|
||||||
|
- **真实日志不足**：先使用可解释的合成场景，明确标注参数来源，再逐步替换为实测参数；
|
||||||
|
- **大规模运行开销过高**：使用事件驱动、稀疏图、节点聚合和分区仿真；
|
||||||
|
- **状态空间过大**：保持通用状态机，具体差异放入能力和参数，而非无限增加状态；
|
||||||
|
- **结果可信度不足**：保留参数来源、置信区间、基线对比和误差分析；
|
||||||
|
- **比赛叙事过于技术化**：用“容量规划、热点定位、重试雪崩预警”贯穿展示。
|
||||||
|
|
||||||
|
# 十、答辩PPT映射建议
|
||||||
|
|
||||||
|
本设计书可以压缩为 15 页答辩 PPT：
|
||||||
|
|
||||||
|
1. 项目背景；2. 核心痛点；3. 建模目标；4. 总体架构；5. 动态通信图；6. 外层状态机；7. 内层状态机；8. 分层参数；9. 离散事件仿真；10. 模拟实验设计流程；11. 输出指标；12. 验证方案；13. 创新点；14. 应用价值；15. 总结与展望。
|
||||||
|
|
||||||
|
# 结论
|
||||||
|
|
||||||
|
本项目以 Agent 的业务行为作为网络流量产生机制，用双层随机状态机连接“任务执行”和“消息传播”，再结合队列、拓扑和离散事件仿真形成可解释、可校准、可扩展的流量预测框架。方案既能服务比赛中的数学建模与仿真验证，也具有容量规划、架构优化和故障预警等工程价值。
|
||||||
@@ -0,0 +1,642 @@
|
|||||||
|
---
|
||||||
|
title: "大规模多智能体网络流量建模方案"
|
||||||
|
subtitle: "基于双层随机混合自动机的离散事件仿真"
|
||||||
|
author: "参赛团队：待填写"
|
||||||
|
date: "2026年8月"
|
||||||
|
toc: true
|
||||||
|
toc-title: "目录"
|
||||||
|
number-sections: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# 本次修订说明
|
||||||
|
|
||||||
|
**修订日期：2026年8月13日**
|
||||||
|
|
||||||
|
本版本补充了基于大模型模拟数据开展快速实验的技术方案，主要改动如下：
|
||||||
|
|
||||||
|
- 在“参数估计与数据方案”章节新增 **7.4 模拟实验设计流程（基于大模型）**；
|
||||||
|
- 定义任务行为画像向量，包括工具调用概率、任务拆分概率、平均子任务数、超时与失败概率、处理时间和消息大小参数；
|
||||||
|
- 说明采用“大模型生成行为画像 + 本地受约束随机引擎生成事件日志”的混合方法，而不是让大模型直接逐行生成海量日志；
|
||||||
|
- 补充 `task_id`、`message_id` 和 `parent_message_id` 驱动的消息因果链构造方法；
|
||||||
|
- 增加概率范围、正值参数、事件时间顺序、消息守恒、训练/验证/测试隔离和随机种子等质量约束；
|
||||||
|
- 补充静态均值基线、无上下文模型、完整双层状态机模型、压力实验和重试放大实验的闭环；
|
||||||
|
- 明确模拟数据的用途是验证模型实现和实验方法，正式结论仍需真实日志或可控 Agent 实验校准。
|
||||||
|
|
||||||
|
上述流程已在 `agent_traffic_experiments/` 中实现，模型输入输出字段和本方案中的状态机、参数估计及评估指标保持对应。
|
||||||
|
|
||||||
|
# 摘要
|
||||||
|
|
||||||
|
本文面向大规模多智能体系统中的通信流量预测问题，建立由动态通信图、双层随机混合自动机、消息队列、路由与目标选择模型、离散事件调度器和流量统计模型组成的统一框架。外层自动机描述消息在节点之间的传播位置和交互路径，内层自动机描述节点接收、排队、思考、任务拆分、工具调用、等待、重试和发送等行为。模型通过节点能力、任务上下文、局部拓扑及系统负载共同决定状态转移概率、状态持续时间、输出消息数量和消息大小。
|
||||||
|
|
||||||
|
参数主要由小规模运行日志、系统配置和压力测试数据估计；对于新任务、新阶段和新节点，采用分层参数共享、特征表示和回退机制实现组合泛化。最终利用离散事件仿真，计算节点与链路流量、任务时延、吞吐量、队列长度、流量放大系数以及故障重试效应，为大规模 Agent 网络的容量规划和架构优化提供依据。
|
||||||
|
|
||||||
|
**关键词：** 多智能体系统；网络流量；随机混合自动机；离散事件仿真；排队模型；分层参数
|
||||||
|
|
||||||
|
# 1 问题定义
|
||||||
|
|
||||||
|
## 1.1 研究对象
|
||||||
|
|
||||||
|
研究对象为由 Agent、工具服务、数据库、模型服务等计算与通信实体构成的异构网络。外部任务进入网络后，节点会根据任务阶段和自身能力进行推理、拆分、协作、查询、转发或返回结果，并产生新的内部消息。
|
||||||
|
|
||||||
|
系统需要根据有限规模日志和配置参数，预测更大规模或更高负载条件下的网络行为。
|
||||||
|
|
||||||
|
## 1.2 输入
|
||||||
|
|
||||||
|
模型输入包括：
|
||||||
|
|
||||||
|
- 网络拓扑及其动态变化规则；
|
||||||
|
- 节点能力、并发上限、服务速度和队列容量；
|
||||||
|
- 外部任务类型、阶段、到达过程和优先级；
|
||||||
|
- 状态转移、状态持续时间和下游消息数量参数；
|
||||||
|
- 不同消息类型的大小分布；
|
||||||
|
- 链路带宽、传播时延和传输失败参数；
|
||||||
|
- 超时阈值、最大重试次数和退避策略。
|
||||||
|
|
||||||
|
## 1.3 输出
|
||||||
|
|
||||||
|
在时间窗口 $T$ 内，输出包括：
|
||||||
|
|
||||||
|
1. 节点入站/出站消息数和字节数；
|
||||||
|
2. 链路累计流量、平均速率和利用率；
|
||||||
|
3. 任务完成时间及平均、P95、P99 延迟；
|
||||||
|
4. 节点队列长度、利用率、吞吐量和丢弃数；
|
||||||
|
5. 平均跳数、内部消息数和流量放大系数；
|
||||||
|
6. 超时率、失败率、重试次数和故障影响范围；
|
||||||
|
7. 热点节点、瓶颈链路和系统稳定区间。
|
||||||
|
|
||||||
|
## 1.4 建模边界
|
||||||
|
|
||||||
|
本模型主要描述应用层消息及其引发的计算、排队和路由行为。若比赛数据只提供逻辑消息大小，则不额外精确建模 TCP/IP 包头、分片和底层重传；如能获得网络层数据，可在逻辑消息大小之上增加协议开销系数。
|
||||||
|
|
||||||
|
# 2 基本假设
|
||||||
|
|
||||||
|
为构建可计算模型，作如下基础假设：
|
||||||
|
|
||||||
|
1. 系统可表示为随时间变化的有向图，节点和链路属性在事件发生时更新；
|
||||||
|
2. 消息是仿真的基本通信对象，每条消息具有来源、目的、类型、大小和任务关联；
|
||||||
|
3. 节点共享通用状态集合，但可执行动作和参数因能力、上下文与负载不同；
|
||||||
|
4. 状态持续时间和消息大小服从从日志估计的经验分布或参数分布；
|
||||||
|
5. 外部任务到达可按实测时间序列重放；无日志时，基线场景采用泊松或非齐次泊松过程；
|
||||||
|
6. 节点处理资源和链路带宽有限，负载超过容量会形成队列或丢弃；
|
||||||
|
7. 超时与失败可触发有限重试，重试策略由最大次数和退避规则控制；
|
||||||
|
8. 不同随机实验使用独立随机种子，并通过多次重复估计均值和置信区间。
|
||||||
|
|
||||||
|
# 3 系统对象与符号定义
|
||||||
|
|
||||||
|
## 3.1 动态通信图
|
||||||
|
|
||||||
|
系统在时刻 $t$ 表示为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
G(t)=(V(t),E(t))
|
||||||
|
$$
|
||||||
|
|
||||||
|
$V(t)$ 为节点集合，$E(t)$ 为有向通信边集合。对链路 $e=(u,v)$，定义：
|
||||||
|
|
||||||
|
$$
|
||||||
|
g_e(t)=(C_e,d_e,u_e(t),p_e^{fail})
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中 $C_e$ 为带宽，$d_e$ 为基础传播时延，$u_e(t)$ 为利用率，$p_e^{fail}$ 为传输失败概率。
|
||||||
|
|
||||||
|
## 3.2 节点
|
||||||
|
|
||||||
|
节点 $v$ 表示为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
v=(x_v,q_v,G_v,z_v,r_v,Q_v)
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中：
|
||||||
|
|
||||||
|
- $x_v$：能力和静态资源向量；
|
||||||
|
- $q_v$：节点级资源状态，取 `Idle`、`Busy`、`Saturated` 或 `Failed`，不表示某条消息所处的业务处理阶段；
|
||||||
|
- $G_v$：局部拓扑子图；
|
||||||
|
- $z_v$：从局部拓扑提取的度数、距离、邻居能力和负载特征；
|
||||||
|
- $r_v$：并发数、处理资源占用及可用容量；
|
||||||
|
- $Q_v$：等待队列。
|
||||||
|
|
||||||
|
能力向量示例：
|
||||||
|
|
||||||
|
```text
|
||||||
|
x_v = {
|
||||||
|
can_reason: true,
|
||||||
|
can_split_task: true,
|
||||||
|
can_call_tool: true,
|
||||||
|
can_query_database: false,
|
||||||
|
can_forward: true,
|
||||||
|
max_concurrency: 4
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 3.3 消息
|
||||||
|
|
||||||
|
消息定义为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
m=(message\_id,task\_id,parent\_message\_id,correlation\_id,
|
||||||
|
attempt\_id,source,destination,next\_hop,message\_type,
|
||||||
|
message\_size,priority,path,h,retry\_count,t_{create})
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中 `destination` 是逻辑最终接收节点，`next_hop` 是本次传输实际到达的下一跳；$h$ 是消息已完成的跳数。`parent_message_id` 用于还原消息触发关系，`task_id` 用于关联同一次任务，`correlation_id` 用于匹配请求、响应、错误和超时，`attempt_id` 用于区分同一逻辑请求的不同发送尝试。
|
||||||
|
|
||||||
|
## 3.4 任务上下文
|
||||||
|
|
||||||
|
任务上下文 $c$ 包含任务类型、任务阶段、复杂度、可靠性要求、实时性要求等。为了支持未知类别，除离散标签外，还可使用如下可解释特征：
|
||||||
|
|
||||||
|
```text
|
||||||
|
long_context, needs_external_tool, collaboration_degree,
|
||||||
|
realtime_requirement, reliability_requirement, result_complexity
|
||||||
|
```
|
||||||
|
|
||||||
|
# 4 双层随机混合自动机
|
||||||
|
|
||||||
|
## 4.1 外层自动机
|
||||||
|
|
||||||
|
外层自动机描述消息的网络位置和交互关系。外层状态可写为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
S_m^{outer}(t)=(v_t,path_m,h_m,c_m)
|
||||||
|
$$
|
||||||
|
|
||||||
|
当节点内部状态转移生成新消息时，外层模型选择目标节点或下一跳，并计算传输完成时间。目标选择模型为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
P(dst=u\mid v,type,c,z_v,\ell)=
|
||||||
|
\frac{\exp(r_u)}{\sum_{k\in\mathcal N_v^{feasible}}\exp(r_k)}
|
||||||
|
$$
|
||||||
|
|
||||||
|
评分 $r_u$ 可以综合能力匹配、跳数、队列长度、链路利用率和历史成功率：
|
||||||
|
|
||||||
|
$$
|
||||||
|
r_u=\theta_1 match_u-\theta_2 distance_u-\theta_3 queue_u
|
||||||
|
-\theta_4 utilization_{vu}+\theta_5 reliability_u
|
||||||
|
$$
|
||||||
|
|
||||||
|
## 4.2 内层自动机
|
||||||
|
|
||||||
|
节点内部状态集合定义为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
\mathcal Q=\{Idle,Receive,Queue,Think,Split,CallAgent,
|
||||||
|
CallTool,Query,Forward,Wait,Retry,Send,Failed\}
|
||||||
|
$$
|
||||||
|
|
||||||
|
该集合表示消息或任务处理上下文状态 $q_m$，不是节点整体资源状态 $q_v$。同一节点可同时维护多个 $q_m$，其数量和资源占用由 $r_v$ 与 `max_concurrency` 约束。
|
||||||
|
|
||||||
|
主要状态转移如下：
|
||||||
|
|
||||||
|
```text
|
||||||
|
Idle → Receive
|
||||||
|
Receive → Queue / Think
|
||||||
|
Queue → Think
|
||||||
|
Think → Send / Split / CallAgent / CallTool / Query / Forward
|
||||||
|
Split / CallAgent / CallTool / Query / Forward → Wait
|
||||||
|
Wait → Think / Retry
|
||||||
|
Retry → Think / Failed
|
||||||
|
Send / Failed → Idle
|
||||||
|
```
|
||||||
|
|
||||||
|
每次状态转移可同时产生状态持续时间、输出消息集合和流量增量。因此，模型属于含离散状态、连续时间和随机输出的混合自动机。
|
||||||
|
|
||||||
|
## 4.3 可行转移过滤
|
||||||
|
|
||||||
|
设状态 $q$ 的候选动作集合为 $\mathcal A(q)$，根据能力、拓扑和资源得到可行集合：
|
||||||
|
|
||||||
|
$$
|
||||||
|
\mathcal A(X)=\{a\in\mathcal A(q):constraint(a,x_v,z_v,r_v,c)=1\}
|
||||||
|
$$
|
||||||
|
|
||||||
|
例如：
|
||||||
|
|
||||||
|
- `can_call_tool=false` 时移除 `CallTool`；
|
||||||
|
- 无可用数据服务时移除 `Query`；
|
||||||
|
- 队列或并发已满时进入等待、拒绝或转发分支；
|
||||||
|
- 达到最大重试次数后移除继续重试分支。
|
||||||
|
|
||||||
|
## 4.4 条件状态转移概率
|
||||||
|
|
||||||
|
对于可行转移 $j$，定义评分：
|
||||||
|
|
||||||
|
$$
|
||||||
|
s_j=\beta_j+\alpha_{task,j}+\gamma_{phase,j}
|
||||||
|
+\eta_{cap,j}+\delta_{topology,j}+\rho_{load,j}
|
||||||
|
$$
|
||||||
|
|
||||||
|
通过 softmax 得到：
|
||||||
|
|
||||||
|
$$
|
||||||
|
P(j\mid X)=\frac{e^{s_j}}
|
||||||
|
{\sum_{k\in\mathcal A(X)}e^{s_k}}
|
||||||
|
$$
|
||||||
|
|
||||||
|
若日志样本较少，可采用带平滑的频率估计作为初始值：
|
||||||
|
|
||||||
|
$$
|
||||||
|
\hat P_{ij}=\frac{N_{ij}+\alpha}{\sum_k N_{ik}+K\alpha}
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中 $\alpha$ 为平滑系数，$K$ 为候选转移数量。
|
||||||
|
|
||||||
|
## 4.5 状态持续时间
|
||||||
|
|
||||||
|
不同状态采用不同持续时间分布。正值且右偏的数据可使用对数正态分布：
|
||||||
|
|
||||||
|
$$
|
||||||
|
\log T_q\sim \mathcal N(
|
||||||
|
\mu_q+\alpha_{task}+\gamma_{phase}+\eta_v+\rho_{load},\sigma_q^2)
|
||||||
|
$$
|
||||||
|
|
||||||
|
当样本量充足时，优先保存经验累积分布，并报告均值、中位数、P95 和 P99，避免只使用平均值掩盖长尾。
|
||||||
|
|
||||||
|
## 4.6 输出消息模型
|
||||||
|
|
||||||
|
一次转移产生的消息数量为 $N_{out}$，消息集合为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
M_{out}=\{m_1,m_2,\ldots,m_{N_{out}}\}
|
||||||
|
$$
|
||||||
|
|
||||||
|
任务拆分的 $N_{out}$ 可采用经验离散分布或泊松、负二项分布。消息大小按类型和阶段建模：
|
||||||
|
|
||||||
|
$$
|
||||||
|
\log S_m\sim\mathcal N(\mu_{type,phase},\sigma_{type,phase}^2)
|
||||||
|
$$
|
||||||
|
|
||||||
|
# 5 队列、链路与时间模型
|
||||||
|
|
||||||
|
## 5.1 节点队列
|
||||||
|
|
||||||
|
节点 $v$ 在时间窗口内的队列动态为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
Q_v(t+\Delta t)=\min\left\{Q_v^{max},
|
||||||
|
\max[0,Q_v(t)+A_v(t)-D_v(t)]\right\}
|
||||||
|
$$
|
||||||
|
|
||||||
|
当队列达到 $Q_v^{max}$ 时，根据系统策略执行丢弃、拒绝、限流或改道。
|
||||||
|
|
||||||
|
$Q_v(t)$ 仅包含已经到达节点但尚未分配到活跃处理资源的消息；$D_v(t)$ 表示窗口内从等待队列取出并开始处理的消息数。队列采用有界、按 `priority` 优先且同优先级按到达顺序处理的规则。默认 `Wait` 上下文仍占用并发槽；实际系统若在等待期间释放资源，应设置 `wait_holds_slot=false`，并在响应到达后重新申请并发槽。
|
||||||
|
|
||||||
|
若某基线场景满足泊松到达和指数服务，可用 M/M/1 结果进行理论校验：
|
||||||
|
|
||||||
|
$$
|
||||||
|
\rho=\frac{\lambda}{\mu},\qquad
|
||||||
|
W=\frac{1}{\mu-\lambda},\qquad \lambda<\mu
|
||||||
|
$$
|
||||||
|
|
||||||
|
正式仿真不强制要求指数分布，可直接使用经验处理时间和多并发服务资源。
|
||||||
|
|
||||||
|
## 5.2 链路传输时间
|
||||||
|
|
||||||
|
消息 $m$ 经过链路 $e$ 的基础传输时间为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
T_{e,m}=d_e+\frac{S_m}{C_e}+T_e^{queue}
|
||||||
|
$$
|
||||||
|
|
||||||
|
如需更细致地表达利用率导致的非线性排队，可设：
|
||||||
|
|
||||||
|
$$
|
||||||
|
T_e^{queue}=\kappa_e\frac{u_e}{1-u_e+\varepsilon}
|
||||||
|
$$
|
||||||
|
|
||||||
|
该形式需要通过压力测试校准，不应在无数据时声称为真实网络规律。
|
||||||
|
|
||||||
|
## 5.3 超时和重试
|
||||||
|
|
||||||
|
请求在超时阈值 $T_{timeout}$ 前未收到有效响应时进入 `Retry`。若单次成功概率为 $1-p$，允许最多 $R$ 次重试，则理论期望请求次数为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
E[N_{request}]=\sum_{k=0}^{R}p^k=
|
||||||
|
\frac{1-p^{R+1}}{1-p}
|
||||||
|
$$
|
||||||
|
|
||||||
|
超时概率本身可以随队列和链路利用率变化：
|
||||||
|
|
||||||
|
$$
|
||||||
|
logit(p_{timeout})=omega_0+omega_1 Q_v+omega_2 u_e+omega_3 T_{service}
|
||||||
|
$$
|
||||||
|
|
||||||
|
# 6 流量与性能指标
|
||||||
|
|
||||||
|
## 6.1 节点流量
|
||||||
|
|
||||||
|
时间窗口 $T$ 内节点入站和出站字节数：
|
||||||
|
|
||||||
|
$$
|
||||||
|
B_v^{in}(T)=\sum_{m:dst(m)=v}S_m,
|
||||||
|
\qquad
|
||||||
|
B_v^{out}(T)=\sum_{m:src(m)=v}S_m
|
||||||
|
$$
|
||||||
|
|
||||||
|
对应平均速率为 $B/T$。
|
||||||
|
|
||||||
|
## 6.2 链路流量
|
||||||
|
|
||||||
|
$$
|
||||||
|
B_e(T)=\sum_{m:e\in path(m)}S_m,
|
||||||
|
\qquad
|
||||||
|
R_e(T)=\frac{B_e(T)}{T}
|
||||||
|
$$
|
||||||
|
|
||||||
|
链路利用率为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
U_e(T)=\frac{R_e(T)}{C_e}
|
||||||
|
$$
|
||||||
|
|
||||||
|
## 6.3 任务级指标
|
||||||
|
|
||||||
|
任务 $i$ 的端到端延迟：
|
||||||
|
|
||||||
|
$$
|
||||||
|
L_i=t_i^{finish}-t_i^{arrival}
|
||||||
|
$$
|
||||||
|
|
||||||
|
任务内部流量放大系数定义为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
AF_i=\frac{\text{任务 }i\text{ 产生的内部总字节数}}
|
||||||
|
{\text{任务 }i\text{ 的外部输入字节数}}
|
||||||
|
$$
|
||||||
|
|
||||||
|
也可分别计算消息数量放大系数、工具调用放大系数和重试放大系数。
|
||||||
|
|
||||||
|
## 6.4 系统稳定性
|
||||||
|
|
||||||
|
对每个节点检查有效到达率与服务能力：
|
||||||
|
|
||||||
|
$$
|
||||||
|
\rho_v=\frac{\lambda_v^{eff}}{\mu_v}
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中 $\mu_v$ 按基准模型定义为节点的总单位时间处理能力，已经综合节点并发槽、资源竞争和消息类型差异，不能再默认乘以 `max_concurrency`。若实测参数是单槽服务率，则必须先根据并发竞争和资源共享关系换算为节点总处理能力。多个关键节点长期满足 $\rho_v\ge 1$ 时，系统通常进入队列持续增长区间。由于任务拆分和重试会改变 $\lambda_v^{eff}$，稳定性需要通过迭代或仿真而非只看外部到达率判断。
|
||||||
|
|
||||||
|
# 7 参数估计与数据方案
|
||||||
|
|
||||||
|
## 7.1 日志模式
|
||||||
|
|
||||||
|
每条事件至少包含：
|
||||||
|
|
||||||
|
| 字段 | 含义 |
|
||||||
|
|---|---|
|
||||||
|
| timestamp | 事件时间 |
|
||||||
|
| task_id、message_id、parent_message_id | 任务与消息触发关系 |
|
||||||
|
| correlation_id、attempt_id | 请求响应匹配与重试尝试区分 |
|
||||||
|
| source、destination、next_hop | 逻辑发送端、逻辑接收端与实际下一跳 |
|
||||||
|
| task_type、task_phase、message_type | 业务上下文 |
|
||||||
|
| state_before、state_after | 状态转移 |
|
||||||
|
| state_duration | 状态持续时间 |
|
||||||
|
| message_size | 消息字节数 |
|
||||||
|
| queue_length、concurrency | 节点负载 |
|
||||||
|
| link_utilization | 链路负载 |
|
||||||
|
| success、retry_count | 结果与重试 |
|
||||||
|
|
||||||
|
## 7.2 估计流程
|
||||||
|
|
||||||
|
1. 根据 task_id 和 parent_message_id 还原任务调用树；
|
||||||
|
2. 校验时间戳顺序、重复消息和缺失字段；
|
||||||
|
3. 按状态、任务、阶段、能力和负载分组；
|
||||||
|
4. 估计转移概率、持续时间和消息大小分布；
|
||||||
|
5. 使用压力测试估计高负载下的服务率和超时率；
|
||||||
|
6. 划分训练集、验证集和测试集；
|
||||||
|
7. 记录每个参数的样本量、来源、版本和置信度。
|
||||||
|
|
||||||
|
## 7.3 未知类别与回退
|
||||||
|
|
||||||
|
运行时依次尝试：
|
||||||
|
|
||||||
|
```text
|
||||||
|
类别专属参数
|
||||||
|
→ 相似类别或特征组合参数
|
||||||
|
→ 同任务类型的上级参数
|
||||||
|
→ 全局状态基础参数
|
||||||
|
```
|
||||||
|
|
||||||
|
回退结果应附带 `parameter_source=fallback` 和较低置信度，避免把缺少数据的预测解释为高可信结论。
|
||||||
|
|
||||||
|
## 7.4 模拟实验设计流程（基于大模型）
|
||||||
|
|
||||||
|
在尚未获得足量真实 Agent 运行日志时，采用“大模型生成行为画像 + 规则约束生成事件日志”的混合数据构造方法。大模型用于提供不同任务类型的语义差异和合理参数组合，本地生成程序负责生成大规模、结构一致且可重复的日志。
|
||||||
|
|
||||||
|
### 7.4.1 流程设计
|
||||||
|
|
||||||
|
```text
|
||||||
|
步骤1：定义任务类型、网络节点和实验负载
|
||||||
|
↓
|
||||||
|
步骤2：大模型生成任务行为画像
|
||||||
|
↓
|
||||||
|
步骤3：执行结构与数值约束校验
|
||||||
|
↓
|
||||||
|
步骤4：本地生成器扩展任务、状态和消息事件
|
||||||
|
↓
|
||||||
|
步骤5：按任务划分训练集、验证集与测试集
|
||||||
|
↓
|
||||||
|
步骤6：训练集估计状态机和分布参数
|
||||||
|
↓
|
||||||
|
步骤7：验证集选择分布和调整参数
|
||||||
|
↓
|
||||||
|
步骤8：测试集作为模拟实测值进行最终预测验证
|
||||||
|
↓
|
||||||
|
步骤9：执行基线、压力和重试实验并输出图表
|
||||||
|
```
|
||||||
|
|
||||||
|
大模型生成的单个行为画像包括：
|
||||||
|
|
||||||
|
$$
|
||||||
|
profile=(p_{tool},p_{split},E[N_{subtask}],p_{timeout},p_{fail},
|
||||||
|
\mu_T,CV_T,\mu_{req},\mu_{resp},CV_S)
|
||||||
|
$$
|
||||||
|
|
||||||
|
分别表示工具调用概率、任务拆分概率、平均子任务数、超时概率、失败概率、处理时间均值和变异系数、请求与响应消息大小均值以及消息大小变异系数。
|
||||||
|
|
||||||
|
### 7.4.2 结构化事件扩展
|
||||||
|
|
||||||
|
行为画像通过本地随机引擎扩展为事件日志。每个任务生成唯一 `task_id`，每条消息生成唯一 `message_id`，并通过 `parent_message_id` 记录任务拆分、工具请求和返回结果之间的因果关系。生成器按照状态机约束生成：
|
||||||
|
|
||||||
|
```text
|
||||||
|
外部任务到达 → 协调节点思考 → 拆分或调用执行节点
|
||||||
|
→ 可选工具调用 → 超时与有限重试 → 执行结果返回
|
||||||
|
→ 协调节点汇总 → 最终响应
|
||||||
|
```
|
||||||
|
|
||||||
|
生成过程中同时维护时间戳、状态持续时间、消息大小、来源节点、目标节点、队列长度、成功状态和重试次数。
|
||||||
|
|
||||||
|
### 7.4.3 数据质量约束
|
||||||
|
|
||||||
|
- 概率参数限制在 $[0,0.95]$，不可行状态转移概率设为零；
|
||||||
|
- 子任务数量、消息大小和处理时间必须为正，并设置合理上限；
|
||||||
|
- 消息标识唯一，父消息必须存在或为空；
|
||||||
|
- 同一任务的事件时间保持因果顺序；
|
||||||
|
- 发送、接收、在途、丢弃和外部输出之间满足消息守恒；
|
||||||
|
- 训练集、验证集和测试集按完整任务划分，避免消息级数据泄漏；
|
||||||
|
- 保存生成模型、提示词版本、配置文件和随机种子。
|
||||||
|
|
||||||
|
### 7.4.4 实验闭环
|
||||||
|
|
||||||
|
训练事件用于估计状态转移概率、持续时间分布、消息大小分布和重试参数；验证事件用于选择分布和调整参数；测试事件只按任务聚合为模拟实测值并用于最终评估。随后分别运行静态均值基线、无任务上下文模型和完整双层状态机模型，并比较内部流量、任务延迟、消息数、工具调用数和流量放大系数。压力实验通过提高外部到达率观察队列和容量拐点，重试实验通过组合超时概率与最大重试次数观察成功率和流量放大。
|
||||||
|
|
||||||
|
该流程的定位是快速验证模型实现、实验方法和指标体系。模拟数据不得表述为真实生产数据；正式结论仍需真实日志或可控 Agent 实验进行校准。
|
||||||
|
|
||||||
|
# 8 离散事件仿真算法
|
||||||
|
|
||||||
|
## 8.1 事件类型
|
||||||
|
|
||||||
|
- ExternalArrival：外部任务到达；
|
||||||
|
- MessageArrival：消息抵达目标节点；
|
||||||
|
- ServiceStart：节点获得处理资源；
|
||||||
|
- StateComplete：状态处理完成；
|
||||||
|
- TransmissionComplete：消息传输完成；
|
||||||
|
- ResponseArrival：协作或工具结果返回；
|
||||||
|
- Timeout：请求超时；
|
||||||
|
- Retry：重新发送或改选目标；
|
||||||
|
- Failure/Recovery：节点或链路故障与恢复；
|
||||||
|
- Snapshot：周期性统计快照。
|
||||||
|
|
||||||
|
## 8.2 核心状态
|
||||||
|
|
||||||
|
仿真器维护：全局时钟、优先事件队列、节点状态、节点等待队列、链路状态、未完成请求表、任务状态表以及流量统计器。
|
||||||
|
|
||||||
|
## 8.3 核心伪代码
|
||||||
|
|
||||||
|
```text
|
||||||
|
初始化拓扑、节点、参数和随机种子
|
||||||
|
生成外部任务到达事件
|
||||||
|
|
||||||
|
while 事件队列非空 且 当前时间 < 仿真终止时间:
|
||||||
|
event = 弹出时间最早的事件
|
||||||
|
clock = event.time
|
||||||
|
|
||||||
|
if event 为消息到达:
|
||||||
|
更新目标节点入站流量
|
||||||
|
若有处理资源则安排处理，否则进入队列
|
||||||
|
|
||||||
|
if event 为状态完成:
|
||||||
|
读取任务、阶段、能力、拓扑和负载
|
||||||
|
过滤不可行转移
|
||||||
|
计算条件概率并抽样下一状态
|
||||||
|
抽样持续时间、输出消息数量和消息大小
|
||||||
|
更新节点状态、队列和资源
|
||||||
|
为输出消息选择目标并安排传输事件
|
||||||
|
必要时安排超时事件
|
||||||
|
|
||||||
|
if event 为响应或超时:
|
||||||
|
取消互斥事件或触发重试/失败
|
||||||
|
|
||||||
|
更新任务、节点、链路和全局指标
|
||||||
|
```
|
||||||
|
|
||||||
|
## 8.4 事件冲突处理
|
||||||
|
|
||||||
|
响应和超时可能同时存在于事件队列。为每个请求维护唯一 `request_id` 和状态标记；先发生的有效事件更新请求状态，后续互斥事件到达时被忽略，防止同一请求既成功又重试。
|
||||||
|
|
||||||
|
## 8.5 可复现性
|
||||||
|
|
||||||
|
所有随机源由统一种子管理。实验配置保存为版本化文件，包含拓扑、参数、仿真时长、预热期、重复次数和种子列表。报告中的每张图应能由对应配置和脚本重新生成。
|
||||||
|
|
||||||
|
# 9 大规模仿真优化
|
||||||
|
|
||||||
|
## 9.1 事件驱动
|
||||||
|
|
||||||
|
不按固定毫秒更新所有节点，只处理实际事件，复杂度主要与事件数量有关。若总事件数为 $M$，二叉堆优先队列的调度复杂度约为 $O(M\log M)$。
|
||||||
|
|
||||||
|
## 9.2 稀疏拓扑与局部查询
|
||||||
|
|
||||||
|
采用邻接表存储通信图，节点选择只遍历可行邻居，避免建立 $|V|^2$ 的全连接矩阵。
|
||||||
|
|
||||||
|
## 9.3 节点聚合
|
||||||
|
|
||||||
|
对于能力、参数和连接模式相同的大量节点，可在宏观实验中按节点群组聚合；在需要尾延迟和热点分析的局部区域保留细粒度仿真。
|
||||||
|
|
||||||
|
## 9.4 分区与并行
|
||||||
|
|
||||||
|
可按网络社区或业务域进行分区，跨区消息作为边界事件交换。并行化时必须保证事件因果顺序，比赛原型阶段可先完成单机确定性版本，再扩展并行实现。
|
||||||
|
|
||||||
|
# 10 三节点算例
|
||||||
|
|
||||||
|
设外部任务大小为 2 KB，路径为：
|
||||||
|
|
||||||
|
```text
|
||||||
|
外部 → Node_A → Node_B → Node_C → Node_B → Node_A
|
||||||
|
```
|
||||||
|
|
||||||
|
其中：
|
||||||
|
|
||||||
|
- A→B 任务请求 3 KB；
|
||||||
|
- B→C 工具请求 1 KB；
|
||||||
|
- C→B 工具响应 4 KB；
|
||||||
|
- B→A 最终结果 2 KB；
|
||||||
|
- B 调用工具的基础概率为 0.7。
|
||||||
|
|
||||||
|
若一次任务实际进入工具调用分支，则节点间链路累计流量为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
B_{AB}=3+2=5\text{ KB}
|
||||||
|
$$
|
||||||
|
|
||||||
|
$$
|
||||||
|
B_{BC}=1+4=5\text{ KB}
|
||||||
|
$$
|
||||||
|
|
||||||
|
若外部任务到达率为每秒 100 条，忽略排队和失败，则期望链路流量近似为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
R_{AB}=100\times5=500\text{ KB/s}
|
||||||
|
$$
|
||||||
|
|
||||||
|
$$
|
||||||
|
R_{BC}=100\times0.7\times5=350\text{ KB/s}
|
||||||
|
$$
|
||||||
|
|
||||||
|
该算例用于验证计数和事件实现。正式实验将引入随机消息大小、处理时间、队列、有限带宽和超时重试。
|
||||||
|
|
||||||
|
# 11 验证设计概述
|
||||||
|
|
||||||
|
模型验证分为：
|
||||||
|
|
||||||
|
1. 手工算例逐事件对账；
|
||||||
|
2. 小规模日志训练/验证/测试集预测；
|
||||||
|
3. 与静态倍数、纯拓扑和 M/M/1 基线比较；
|
||||||
|
4. 移除分层阶段、局部拓扑、队列或重试模块的消融实验；
|
||||||
|
5. 到达率、带宽、并发数、超时率和重试上限敏感性分析；
|
||||||
|
6. 10—100000 仿真节点的性能扩展实验。
|
||||||
|
|
||||||
|
预测误差使用 MAE、RMSE 和 MAPE；尾延迟单独比较 P95/P99；热点识别使用 Precision、Recall 和 F1；仿真性能报告运行时间、内存峰值和每秒处理事件数。
|
||||||
|
|
||||||
|
# 12 模型局限与改进方向
|
||||||
|
|
||||||
|
- 小规模日志不能直接覆盖大规模系统中的新拥塞机制，需要压力实验校准；
|
||||||
|
- 泊松到达只适合作为无实测数据时的基线，突发任务应使用实测序列或批量到达模型；
|
||||||
|
- 参数之间可能存在相关性，独立抽样会低估极端事件，后续可引入联合分布或条件生成模型；
|
||||||
|
- 若新能力引入流式通信、广播或长期连接，需要扩展状态机结构；
|
||||||
|
- 超大规模精细仿真计算成本较高，可研究多分辨率和代理模型加速；
|
||||||
|
- 模型预测代表给定假设和参数下的仿真结果，必须同步报告参数来源和置信度。
|
||||||
|
|
||||||
|
# 13 最终模型定义
|
||||||
|
|
||||||
|
消息处理事件在时刻 $t$ 的完整状态表示为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
s_m(t)=(v,q_v,q_m,x_v,z_v,c,m,path,h,Q_v(t),r_v,\tau_v,retry\_count)
|
||||||
|
$$
|
||||||
|
|
||||||
|
一次状态更新为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
(s_{t+\Delta t},M_{out},\Delta F)=f(s_t,g_e(t),\xi_t)
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中 $\xi_t$ 表示状态选择、持续时间、消息大小、目标选择和故障等随机变量，$\Delta F$ 表示节点及链路流量增量。
|
||||||
|
|
||||||
|
因此，整体模型可概括为：
|
||||||
|
|
||||||
|
$$
|
||||||
|
\boxed{\text{随机混合自动机}+\text{消息队列}+\text{动态通信图}
|
||||||
|
+\text{事件调度器}+\text{路由模型}+\text{流量统计}}
|
||||||
|
$$
|
||||||
|
|
||||||
|
# 结论
|
||||||
|
|
||||||
|
本方案将 Agent 的任务行为转化为可观测、可估计的消息生成机制，并通过双层状态机与动态网络连接起来。模型既保留状态、路径、参数和流量之间的可解释关系，又能通过事件仿真表达队列、拥塞、超时和重试的非线性反馈。下一阶段应以最小可运行仿真器和标准化实验为重点，用数据检验模型精度、模块必要性和大规模运行能力。
|
||||||
@@ -0,0 +1,554 @@
|
|||||||
|
---
|
||||||
|
title: "大规模多智能体网络流量模型"
|
||||||
|
subtitle: "验证与评估实验计划（待执行）"
|
||||||
|
author: "参赛团队：待填写"
|
||||||
|
date: "2026年8月"
|
||||||
|
toc: true
|
||||||
|
toc-title: "目录"
|
||||||
|
number-sections: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# 本次修订说明
|
||||||
|
|
||||||
|
**修订日期：2026年8月13日**
|
||||||
|
|
||||||
|
本版本在实验总体框架中加入了可立即执行的模拟实验方案，主要改动如下：
|
||||||
|
|
||||||
|
- 在“验证总体框架”后新增 **1.3 模拟实验设计流程**；
|
||||||
|
- 将模拟过程拆分为场景定义、LLM 行为画像生成、画像校验、结构化日志生成、训练/验证/测试隔离、参数估计、快速实验和结果输出八个环节；
|
||||||
|
- 设置简单问答、工具研究和多 Agent 协作分析三类初始任务场景；
|
||||||
|
- 规定 LLM 输出固定 JSON 画像，并在缺少 API Key 或请求失败时使用版本化本地画像回退；
|
||||||
|
- 明确训练集用于估计参数、验证集用于选择分布和调整参数、测试集只用于最终评估，避免任务级数据泄漏；
|
||||||
|
- 增加预测验证、模型基线、到达率压力和超时重试四类快速实验的输入输出说明；
|
||||||
|
- 规定保存原始事件 CSV、行为画像 JSON、估计参数 JSON、实验结果 CSV、图表、配置文件和随机种子；
|
||||||
|
- 强调最终报告必须把模拟结果标注为“模拟数据”，获得真实日志后使用同一实验管线重新验证。
|
||||||
|
|
||||||
|
配套实验工程位于 `agent_traffic_experiments/`，当前已经能够自动生成数据并输出预测对比图、压力曲线和重试流量放大热力图。
|
||||||
|
|
||||||
|
# 文档说明
|
||||||
|
|
||||||
|
本文是验证与评估报告的前置计划，不包含尚未实际获得的准确率、性能或显著性结论。实验执行后，应将本计划中的“预期图表、数据表和验收标准”替换或补充为真实结果，并保留失败实验和误差解释。
|
||||||
|
|
||||||
|
实验目标是回答四个问题：
|
||||||
|
|
||||||
|
1. 仿真程序是否正确实现了数学模型？
|
||||||
|
2. 模型能否预测真实或半真实 Agent 网络的流量与延迟？
|
||||||
|
3. 双层状态机、队列、拓扑和重试模块是否确有必要？
|
||||||
|
4. 模型能否在更大规模网络中保持可接受的运行开销与稳定性？
|
||||||
|
|
||||||
|
# 1 验证总体框架
|
||||||
|
|
||||||
|
## 1.1 验证层次
|
||||||
|
|
||||||
|
| 层次 | 核心问题 | 主要方法 | 输出 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| V1 实现正确性 | 事件、消息和流量是否算对 | 手工算例、单元测试、守恒检查 | 对账表 |
|
||||||
|
| V2 参数可信度 | 参数是否由日志稳定估计 | 分布拟合、Bootstrap、训练/验证/测试划分 | 参数表与区间 |
|
||||||
|
| V3 预测准确性 | 能否预测测试场景 | 实测或重放对比 | 误差指标与拟合图 |
|
||||||
|
| V4 模型有效性 | 完整模型是否优于简化模型 | 基线和消融实验 | 对比表 |
|
||||||
|
| V5 鲁棒与扩展性 | 高负载、故障和大规模下表现如何 | 压力、故障、敏感性和规模实验 | 容量与性能曲线 |
|
||||||
|
|
||||||
|
## 1.2 实验原则
|
||||||
|
|
||||||
|
- 所有实验配置、随机种子和软件版本可追踪；
|
||||||
|
- 训练数据不得进入测试集；
|
||||||
|
- 每个随机场景至少重复多次，并报告均值和 95% 置信区间；
|
||||||
|
- 真实数据、合成数据和假设参数必须清楚标注；
|
||||||
|
- 不仅报告平均值，还报告 P95、P99 和最差场景；
|
||||||
|
- 不删除对模型不利的异常结果，应分析其原因和适用边界。
|
||||||
|
|
||||||
|
## 1.3 模拟实验设计流程
|
||||||
|
|
||||||
|
在真实日志尚不足以覆盖全部任务类型、网络规模和故障条件时，先通过受约束的大模型模拟构建实验数据，快速验证模型与代码闭环。流程如下。
|
||||||
|
|
||||||
|
```text
|
||||||
|
任务与场景设计
|
||||||
|
↓
|
||||||
|
LLM生成行为画像
|
||||||
|
↓
|
||||||
|
画像合法性校验与默认参数回退
|
||||||
|
↓
|
||||||
|
本地随机引擎扩展结构化事件日志
|
||||||
|
↓
|
||||||
|
按任务划分训练集 / 验证集 / 测试集
|
||||||
|
↓
|
||||||
|
训练集估计状态机参数
|
||||||
|
↓
|
||||||
|
验证集选择分布和调整参数
|
||||||
|
↓
|
||||||
|
测试集聚合为模拟实测值
|
||||||
|
↓
|
||||||
|
完整模型、无上下文模型、静态均值基线对比
|
||||||
|
↓
|
||||||
|
压力实验与重试放大实验
|
||||||
|
↓
|
||||||
|
自动输出数据表、参数文件、指标和图表
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.3.1 第一步：定义模拟场景
|
||||||
|
|
||||||
|
首轮至少设置三类具有复杂度梯度的任务：
|
||||||
|
|
||||||
|
| 任务类型 | 主要特点 | 预期通信行为 |
|
||||||
|
|---|---|---|
|
||||||
|
| 简单问答 | 单节点推理为主 | 消息少、时延短、很少调用工具 |
|
||||||
|
| 工具研究 | 需要搜索或数据查询 | 工具调用率高、响应消息较大 |
|
||||||
|
| 多 Agent 协作分析 | 任务拆分和结果汇总 | 子任务多、路径长、队列和重试影响明显 |
|
||||||
|
|
||||||
|
同时定义 Agent 数量、工具数量、并发容量、队列容量、链路带宽和基础时延。
|
||||||
|
|
||||||
|
### 1.3.2 第二步：LLM生成行为画像
|
||||||
|
|
||||||
|
大模型不直接输出海量日志，而是为每类任务生成有限的行为参数：工具调用概率、拆分概率、平均子任务数、处理时间、请求/响应大小、超时率和失败率。输出必须采用固定 JSON 字段，并经过范围检查。
|
||||||
|
|
||||||
|
若没有 API Key 或模型调用失败，使用版本化的内置画像回退，以保证实验可离线复现；报告中记录本次实际使用的是 LLM 画像还是本地回退画像。
|
||||||
|
|
||||||
|
### 1.3.3 第三步：生成结构化日志
|
||||||
|
|
||||||
|
本地程序根据画像运行受约束的随机状态机，为每个任务生成外部到达、Agent 请求、工具请求、工具响应、Agent 返回、超时、重试和最终响应事件。日志必须保留：
|
||||||
|
|
||||||
|
- 任务 ID、消息 ID、父消息 ID、请求响应关联 ID 和发送尝试 ID；
|
||||||
|
- 任务类型、任务阶段和消息类型；
|
||||||
|
- 来源节点、逻辑目标节点、实际下一跳和状态转移；
|
||||||
|
- 时间戳、状态持续时间和队列长度；
|
||||||
|
- 消息大小、成功状态和重试次数。
|
||||||
|
|
||||||
|
### 1.3.4 第四步：训练与测试隔离
|
||||||
|
|
||||||
|
按完整任务划分训练集、验证集与测试集。训练集只用于估计参数；验证集用于选择持续时间和消息大小分布、调整模型参数；测试集不得用于调参，只按任务聚合得到内部字节数、端到端延迟、消息数、工具调用数、重试数和成功率，作为最终模拟实验的对照值。
|
||||||
|
|
||||||
|
### 1.3.5 第五步：快速实验
|
||||||
|
|
||||||
|
| 实验 | 自变量 | 主要输出 |
|
||||||
|
|---|---|---|
|
||||||
|
| 预测验证 | 任务类型 | 流量、延迟、消息数预测误差 |
|
||||||
|
| 基线对比 | 模型版本 | MAE、RMSE、MAPE |
|
||||||
|
| 压力实验 | 外部任务到达率 | 吞吐量、P95延迟、峰值队列、丢弃率 |
|
||||||
|
| 重试实验 | 超时概率、重试上限 | 成功率、平均重试数、流量放大系数 |
|
||||||
|
|
||||||
|
### 1.3.6 第六步：结果输出与使用边界
|
||||||
|
|
||||||
|
程序自动输出原始事件 CSV、行为画像 JSON、估计参数 JSON、实验结果 CSV 和图表。模拟实验主要用于验证模型机制、筛选关键参数和形成比赛报告的初步图表，所有结果必须标注“模拟数据”。获得真实运行日志后，保持相同实验管线，用真实数据重新估参和复验。
|
||||||
|
|
||||||
|
# 2 实验环境与复现规范
|
||||||
|
|
||||||
|
## 2.1 待记录环境
|
||||||
|
|
||||||
|
| 项目 | 记录内容 |
|
||||||
|
|---|---|
|
||||||
|
| 硬件 | CPU型号、核数、内存、操作系统 |
|
||||||
|
| 软件 | Python及依赖版本、仿真器提交版本 |
|
||||||
|
| 配置 | 拓扑文件、参数文件、任务场景文件 |
|
||||||
|
| 随机性 | 主随机种子、重复实验种子列表 |
|
||||||
|
| 运行 | 开始时间、结束时间、预热期、仿真时长 |
|
||||||
|
| 输出 | 原始事件日志、聚合指标、图表脚本 |
|
||||||
|
|
||||||
|
## 2.2 目录建议
|
||||||
|
|
||||||
|
```text
|
||||||
|
experiments/
|
||||||
|
configs/ # 场景、拓扑和参数
|
||||||
|
raw_logs/ # 原始Agent或仿真事件日志
|
||||||
|
processed/ # 清洗后的标准数据
|
||||||
|
scripts/ # 运行、统计和绘图脚本
|
||||||
|
results/ # 每次实验的机器可读结果
|
||||||
|
figures/ # 报告图表
|
||||||
|
manifests/ # 环境、版本、种子与校验信息
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2.3 数据划分
|
||||||
|
|
||||||
|
如有真实任务日志，建议按任务而非单条消息划分，避免同一任务的消息同时出现在训练集和测试集。
|
||||||
|
|
||||||
|
- 训练集：60%，用于参数估计；
|
||||||
|
- 验证集：20%，用于选择分布和超参数；
|
||||||
|
- 测试集：20%，只用于最终评估。
|
||||||
|
|
||||||
|
若数据具有明显时间漂移，应采用前段训练、后段测试的时间切分，并额外报告随机切分结果。
|
||||||
|
|
||||||
|
# 3 指标体系
|
||||||
|
|
||||||
|
## 3.1 预测误差
|
||||||
|
|
||||||
|
对节点流量、链路流量、吞吐量和平均延迟计算：
|
||||||
|
|
||||||
|
$$
|
||||||
|
MAE=\frac{1}{n}\sum_{i=1}^{n}|\hat y_i-y_i|
|
||||||
|
$$
|
||||||
|
|
||||||
|
$$
|
||||||
|
RMSE=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(\hat y_i-y_i)^2}
|
||||||
|
$$
|
||||||
|
|
||||||
|
$$
|
||||||
|
MAPE=\frac{100\%}{n}\sum_{i=1}^{n}
|
||||||
|
\left|\frac{\hat y_i-y_i}{y_i+\varepsilon}\right|
|
||||||
|
$$
|
||||||
|
|
||||||
|
对于真实值接近零的对象，MAPE 不稳定，应同时报告 MAE、SMAPE 或加权 MAPE。
|
||||||
|
|
||||||
|
## 3.2 分布与尾部指标
|
||||||
|
|
||||||
|
- 平均延迟、中位数、P90、P95、P99；
|
||||||
|
- 队列长度分布及最大值；
|
||||||
|
- 消息大小和状态持续时间分布距离；
|
||||||
|
- 可选使用 KS 统计量或 Wasserstein 距离比较分布。
|
||||||
|
|
||||||
|
## 3.3 热点识别
|
||||||
|
|
||||||
|
将利用率或流量处于前 $k\%$ 的节点/链路定义为热点，计算 Precision、Recall、F1 和 Top-K 命中率。
|
||||||
|
|
||||||
|
## 3.4 仿真性能
|
||||||
|
|
||||||
|
- 总运行时间；
|
||||||
|
- 峰值内存；
|
||||||
|
- 每秒处理事件数；
|
||||||
|
- 单任务平均事件数；
|
||||||
|
- 节点规模增加时的时间与内存增长率。
|
||||||
|
|
||||||
|
## 3.5 稳定性与可靠性
|
||||||
|
|
||||||
|
- 任务成功率和失败率；
|
||||||
|
- 超时率和平均重试次数；
|
||||||
|
- 流量放大系数；
|
||||||
|
- 队列是否在仿真后段持续增长；
|
||||||
|
- 故障恢复时间和受影响任务比例。
|
||||||
|
|
||||||
|
# 4 实验E1：三节点手工算例与单元验证
|
||||||
|
|
||||||
|
## 4.1 目的
|
||||||
|
|
||||||
|
验证消息生成、状态转换、链路累计、节点收发流量、超时取消和重试计数是否正确。
|
||||||
|
|
||||||
|
## 4.2 场景
|
||||||
|
|
||||||
|
```text
|
||||||
|
外部 → A → B → C → B → A
|
||||||
|
```
|
||||||
|
|
||||||
|
消息大小依次为 2、3、1、4、2 KB。关闭随机性并固定所有处理时间。分别运行：
|
||||||
|
|
||||||
|
1. 正常工具调用；
|
||||||
|
2. 不调用工具直接返回；
|
||||||
|
3. 第一次工具调用超时、第二次成功；
|
||||||
|
4. 超过最大重试次数并失败；
|
||||||
|
5. B 节点无处理资源，消息进入队列。
|
||||||
|
|
||||||
|
## 4.3 检查项
|
||||||
|
|
||||||
|
- 节点 A/B/C 入站和出站消息数；
|
||||||
|
- A-B、B-C 链路累计字节数；
|
||||||
|
- 消息路径和跳数；
|
||||||
|
- `destination` 与逐跳 `next_hop` 的一致性；
|
||||||
|
- 队列入队、出队和并发资源释放；
|
||||||
|
- 节点资源状态 $q_v$ 与各消息处理上下文状态 $q_m$ 的一致性；
|
||||||
|
- 响应成功后对应超时事件失效；
|
||||||
|
- 任务完成或失败后不存在悬挂请求。
|
||||||
|
|
||||||
|
## 4.4 通过标准
|
||||||
|
|
||||||
|
确定性计数应与手工结果完全一致；浮点时间误差应低于预设容差；所有守恒检查通过。
|
||||||
|
|
||||||
|
# 5 实验E2：参数估计与分布拟合
|
||||||
|
|
||||||
|
## 5.1 目的
|
||||||
|
|
||||||
|
检验从小规模日志估计转移概率、持续时间、消息大小和失败参数的稳定性。
|
||||||
|
|
||||||
|
## 5.2 步骤
|
||||||
|
|
||||||
|
1. 清洗并按任务还原调用链；
|
||||||
|
2. 统计各状态的转移计数和样本量；
|
||||||
|
3. 对持续时间和消息大小比较经验分布、对数正态、Gamma 等候选；
|
||||||
|
4. 使用验证集选择分布；
|
||||||
|
5. 对参数进行 Bootstrap，计算 95% 置信区间；
|
||||||
|
6. 检查任务类型、阶段和负载分层后的样本稀疏问题；
|
||||||
|
7. 为低样本组启用平滑或上级参数回退。
|
||||||
|
|
||||||
|
## 5.3 输出表
|
||||||
|
|
||||||
|
| 参数 | 分组条件 | 样本量 | 估计值/分布 | 95%区间 | 来源 | 置信度 |
|
||||||
|
|---|---|---:|---|---|---|---|
|
||||||
|
| P(Think→CallTool) | 待填写 | | | | 真实/合成 | |
|
||||||
|
| T_Think | 待填写 | | | | 真实/合成 | |
|
||||||
|
| S_request | 待填写 | | | | 真实/合成 | |
|
||||||
|
| p_timeout | 待填写 | | | | 压测/日志 | |
|
||||||
|
|
||||||
|
## 5.4 判定原则
|
||||||
|
|
||||||
|
不预设必须选择某种理论分布。若参数分布拟合较差，正式仿真使用经验抽样，并在报告中说明样本覆盖范围。
|
||||||
|
|
||||||
|
# 6 实验E3：小规模预测准确性
|
||||||
|
|
||||||
|
## 6.1 目的
|
||||||
|
|
||||||
|
使用训练集估计参数，在未参与估参的测试任务上预测流量、延迟和调用次数。
|
||||||
|
|
||||||
|
## 6.2 场景建议
|
||||||
|
|
||||||
|
- 节点规模：3、5、10；
|
||||||
|
- 任务类型：至少 2 类；
|
||||||
|
- 任务阶段：简单任务与工具密集任务；
|
||||||
|
- 负载：低、中、高三个档位；
|
||||||
|
- 每个场景包含足够任务，并运行多次随机仿真。
|
||||||
|
|
||||||
|
## 6.3 比较对象
|
||||||
|
|
||||||
|
- 各节点消息率和字节率；
|
||||||
|
- 各链路累计流量；
|
||||||
|
- 平均、P95、P99 延迟；
|
||||||
|
- 平均工具调用数、下游消息数和重试数；
|
||||||
|
- 吞吐量、失败率和平均队列长度。
|
||||||
|
|
||||||
|
## 6.4 预期图表
|
||||||
|
|
||||||
|
1. 预测值—实测值散点图及 $y=x$ 参考线；
|
||||||
|
2. 各节点流量误差条形图；
|
||||||
|
3. 实测与预测延迟累积分布曲线；
|
||||||
|
4. 不同负载下的 MAPE/MAE 对比；
|
||||||
|
5. 任务级流量放大系数箱线图。
|
||||||
|
|
||||||
|
## 6.5 初步验收目标
|
||||||
|
|
||||||
|
在没有比赛官方阈值时，不应预先承诺固定精度。可使用以下内部目标推动迭代：完整模型在多数主要指标上优于所有基线；测试误差的置信区间稳定；高负载误差上升能够得到合理解释。最终报告填写真实数值。
|
||||||
|
|
||||||
|
# 7 实验E4：基线模型对比
|
||||||
|
|
||||||
|
## 7.1 基线定义
|
||||||
|
|
||||||
|
| 编号 | 基线 | 描述 |
|
||||||
|
|---|---|---|
|
||||||
|
| B0 | 静态平均倍数 | 外部流量乘以固定放大系数 |
|
||||||
|
| B1 | 纯拓扑随机游走 | 仅按连接和固定路由概率传播 |
|
||||||
|
| B2 | 简单排队模型 | 到达率和服务率驱动，不表达任务状态 |
|
||||||
|
| B3 | 单层状态机 | 节点行为和跨节点传播不分层 |
|
||||||
|
| M | 完整模型 | 双层状态机、分层参数、队列和重试闭环 |
|
||||||
|
|
||||||
|
## 7.2 公平性要求
|
||||||
|
|
||||||
|
- 各模型使用相同训练任务和测试任务；
|
||||||
|
- 可共享的外部到达率、平均消息大小和节点总处理能力 $\mu$ 保持一致；
|
||||||
|
- 不允许完整模型使用测试集参数；
|
||||||
|
- 同时比较精度和运行开销，避免只比较预测误差。
|
||||||
|
|
||||||
|
## 7.3 结果表模板
|
||||||
|
|
||||||
|
| 模型 | 节点流量MAPE | 链路流量MAPE | 平均延迟误差 | P95误差 | 运行时间 |
|
||||||
|
|---|---:|---:|---:|---:|---:|
|
||||||
|
| B0 | | | | | |
|
||||||
|
| B1 | | | | | |
|
||||||
|
| B2 | | | | | |
|
||||||
|
| B3 | | | | | |
|
||||||
|
| M | | | | | |
|
||||||
|
|
||||||
|
# 8 实验E5：消融实验
|
||||||
|
|
||||||
|
## 8.1 消融项
|
||||||
|
|
||||||
|
| 消融版本 | 移除内容 | 要验证的假设 |
|
||||||
|
|---|---|---|
|
||||||
|
| A1 | 移除任务阶段修正 | 阶段信息能提升行为预测 |
|
||||||
|
| A2 | 移除局部拓扑和邻居负载 | 拓扑负载影响目标选择与热点 |
|
||||||
|
| A3 | 使用固定平均处理时间 | 长尾分布影响尾延迟 |
|
||||||
|
| A4 | 移除节点队列 | 队列是高负载延迟的关键来源 |
|
||||||
|
| A5 | 移除失败与重试 | 重试影响流量放大与稳定性 |
|
||||||
|
| A6 | 固定节点类型替代能力向量 | 能力表示改善异构节点泛化 |
|
||||||
|
|
||||||
|
## 8.2 分析方式
|
||||||
|
|
||||||
|
比较完整模型与各消融版本在低、中、高负载下的误差变化。若某模块对所有场景几乎没有贡献，应检查参数是否未被正确使用，或重新评估模块复杂度是否值得保留。
|
||||||
|
|
||||||
|
# 9 实验E6：压力与容量拐点
|
||||||
|
|
||||||
|
## 9.1 自变量
|
||||||
|
|
||||||
|
逐步提高外部到达率：
|
||||||
|
|
||||||
|
$$
|
||||||
|
\lambda\in\{0.2,0.4,0.6,0.8,1.0,1.2,1.5\}\times C_{baseline}
|
||||||
|
$$
|
||||||
|
|
||||||
|
其中 $C_{baseline}$ 是基准系统的估计处理能力。每个负载档运行足够长的预热期和统计期。
|
||||||
|
|
||||||
|
## 9.2 观测指标
|
||||||
|
|
||||||
|
- 吞吐量；
|
||||||
|
- 平均、P95、P99 延迟；
|
||||||
|
- 关键节点队列长度；
|
||||||
|
- 超时率、失败率和重试率；
|
||||||
|
- 节点与链路利用率；
|
||||||
|
- 流量放大系数。
|
||||||
|
|
||||||
|
## 9.3 容量拐点定义
|
||||||
|
|
||||||
|
可结合以下现象定义容量拐点：吞吐量不再随到达率线性增长；队列在统计期持续增长；P95 延迟突增；失败率超过业务阈值；重试导致内部流量明显非线性增加。
|
||||||
|
|
||||||
|
## 9.4 预期图表
|
||||||
|
|
||||||
|
- 到达率—吞吐量曲线；
|
||||||
|
- 到达率—P95/P99 延迟曲线；
|
||||||
|
- 时间—队列长度曲线；
|
||||||
|
- 到达率—超时率/重试率曲线；
|
||||||
|
- 到达率—流量放大系数曲线。
|
||||||
|
|
||||||
|
# 10 实验E7：故障与重试放大
|
||||||
|
|
||||||
|
## 10.1 场景矩阵
|
||||||
|
|
||||||
|
| 因素 | 建议水平 |
|
||||||
|
|---|---|
|
||||||
|
| 节点故障比例 | 0%、1%、5%、10% |
|
||||||
|
| 链路带宽下降 | 0%、25%、50%、75% |
|
||||||
|
| 基础超时概率 | 0、0.02、0.05、0.10、0.20 |
|
||||||
|
| 最大重试次数 | 0、1、2、3、5 |
|
||||||
|
| 退避策略 | 无退避、固定退避、指数退避 |
|
||||||
|
| 目标选择 | 固定节点、负载感知、故障感知 |
|
||||||
|
|
||||||
|
## 10.2 关键问题
|
||||||
|
|
||||||
|
1. 哪种重试上限在成功率和额外流量之间更均衡？
|
||||||
|
2. 指数退避能否减轻拥塞雪崩？
|
||||||
|
3. 负载感知或故障感知路由能否缩小影响范围？
|
||||||
|
4. 哪些热点节点故障会造成最大任务失败率？
|
||||||
|
|
||||||
|
## 10.3 结果展示
|
||||||
|
|
||||||
|
使用热力图展示“超时概率 × 重试上限”对成功率和流量放大系数的影响；使用拓扑图展示故障前后的热点迁移；使用时间曲线展示拥塞与恢复过程。
|
||||||
|
|
||||||
|
# 11 实验E8：拓扑和调度策略对比
|
||||||
|
|
||||||
|
## 11.1 拓扑
|
||||||
|
|
||||||
|
- 星型：中心协调节点连接所有执行节点；
|
||||||
|
- 树型：分层协调和任务拆分；
|
||||||
|
- 随机稀疏图：一般协作网络；
|
||||||
|
- 小世界：高聚类、少量远程连接；
|
||||||
|
- 无标度图：少量枢纽节点拥有高连接度。
|
||||||
|
|
||||||
|
## 11.2 策略
|
||||||
|
|
||||||
|
- 最短跳数；
|
||||||
|
- 随机可行邻居；
|
||||||
|
- 最短队列；
|
||||||
|
- 能力匹配优先；
|
||||||
|
- 综合能力、距离、负载和可靠性的加权策略。
|
||||||
|
|
||||||
|
## 11.3 指标
|
||||||
|
|
||||||
|
比较平均跳数、流量集中度、最大节点利用率、任务延迟、成功率和重路由次数。重点讨论不同拓扑是否会形成单点瓶颈，以及负载感知策略是否以额外跳数换取更低尾延迟。
|
||||||
|
|
||||||
|
# 12 实验E9：规模扩展与计算性能
|
||||||
|
|
||||||
|
## 12.1 规模设置
|
||||||
|
|
||||||
|
```text
|
||||||
|
10、100、1 000、10 000、100 000 个仿真节点
|
||||||
|
```
|
||||||
|
|
||||||
|
对每个规模控制平均度数、任务到达率与节点数量的比例，并分别报告低负载和中负载结果。若 100000 节点无法在现有硬件完成，应如实报告达到的最大规模、瓶颈和优化方向。
|
||||||
|
|
||||||
|
## 12.2 测量
|
||||||
|
|
||||||
|
- 初始化时间；
|
||||||
|
- 仿真运行时间；
|
||||||
|
- 峰值内存；
|
||||||
|
- 每秒事件数；
|
||||||
|
- 事件总数；
|
||||||
|
- 结果聚合时间；
|
||||||
|
- 不同规模下的预测指标稳定性。
|
||||||
|
|
||||||
|
## 12.3 对比版本
|
||||||
|
|
||||||
|
如实现条件允许，对比：
|
||||||
|
|
||||||
|
1. 固定时间步与离散事件；
|
||||||
|
2. 全量节点与同构节点聚合；
|
||||||
|
3. 不同优先队列实现；
|
||||||
|
4. 单线程与分区并行版本。
|
||||||
|
|
||||||
|
# 13 实验E10：敏感性与不确定性分析
|
||||||
|
|
||||||
|
## 13.1 关键参数
|
||||||
|
|
||||||
|
- 工具调用概率；
|
||||||
|
- 下游任务数量；
|
||||||
|
- 消息大小均值与方差；
|
||||||
|
- 节点处理速度；
|
||||||
|
- 链路带宽和时延；
|
||||||
|
- 超时概率与重试上限；
|
||||||
|
- 外部任务突发程度。
|
||||||
|
|
||||||
|
## 13.2 方法
|
||||||
|
|
||||||
|
第一阶段使用单因素局部敏感性分析；第二阶段可使用拉丁超立方抽样或 Sobol 方法分析全局敏感性。对高不确定参数从其估计区间中抽样，输出预测指标的置信区间，而不是只给单点预测。
|
||||||
|
|
||||||
|
## 13.3 输出
|
||||||
|
|
||||||
|
- 参数敏感性排序；
|
||||||
|
- 龙卷风图；
|
||||||
|
- 参数变化与输出变化曲线；
|
||||||
|
- 预测区间随样本量的变化；
|
||||||
|
- 最需要补采数据的参数列表。
|
||||||
|
|
||||||
|
# 14 数据质量与守恒检查
|
||||||
|
|
||||||
|
每次实验自动执行以下检查：
|
||||||
|
|
||||||
|
- 每条消息有且仅有一个 task_id 和 message_id；
|
||||||
|
- 除外部输入、最终输出和丢弃外，发送消息数与接收/在途消息数守恒；
|
||||||
|
- 节点并发数不超过上限，队列长度不为负；
|
||||||
|
- 链路累计字节数等于经过该链路消息大小之和；
|
||||||
|
- 已完成请求不会再次触发有效超时；
|
||||||
|
- 任务完成后未完成子请求数为零或被明确标记为取消；
|
||||||
|
- 聚合指标能够由原始事件日志重新计算。
|
||||||
|
|
||||||
|
# 15 报告图表清单
|
||||||
|
|
||||||
|
最终《验证与评估报告》至少包含：
|
||||||
|
|
||||||
|
1. 实验环境与数据集统计表；
|
||||||
|
2. 参数估计及置信区间表；
|
||||||
|
3. 三节点手工对账表；
|
||||||
|
4. 完整模型与基线的误差对比表；
|
||||||
|
5. 预测值—实测值散点图；
|
||||||
|
6. 延迟 CDF 或分位数对比图；
|
||||||
|
7. 消融实验条形图；
|
||||||
|
8. 到达率—吞吐量—尾延迟曲线；
|
||||||
|
9. 超时率—重试次数—流量放大热力图；
|
||||||
|
10. 拓扑热点图；
|
||||||
|
11. 节点规模—运行时间/内存曲线；
|
||||||
|
12. 敏感性排序图；
|
||||||
|
13. 失败案例及误差来源表。
|
||||||
|
|
||||||
|
# 16 执行排期
|
||||||
|
|
||||||
|
| 周期 | 任务 | 完成判据 |
|
||||||
|
|---|---|---|
|
||||||
|
| 第1阶段 | 仿真器最小闭环与E1 | 手工算例全部通过 |
|
||||||
|
| 第2阶段 | 日志模式、合成数据与E2 | 参数表可自动生成 |
|
||||||
|
| 第3阶段 | E3基准预测与E4基线 | 获得第一版误差结果 |
|
||||||
|
| 第4阶段 | E5消融与E6压力 | 明确模块贡献和容量拐点 |
|
||||||
|
| 第5阶段 | E7故障、E8拓扑 | 得到重试与路由结论 |
|
||||||
|
| 第6阶段 | E9规模、E10敏感性 | 完成性能和不确定性分析 |
|
||||||
|
| 第7阶段 | 报告整合与复现检查 | 图表可一键复现、结论有数据支撑 |
|
||||||
|
|
||||||
|
# 17 最终报告写作模板
|
||||||
|
|
||||||
|
正式报告建议按以下顺序组织：
|
||||||
|
|
||||||
|
1. 验证目标与实验环境；
|
||||||
|
2. 数据来源、清洗和参数估计；
|
||||||
|
3. 实现正确性验证；
|
||||||
|
4. 预测准确性与基线对比；
|
||||||
|
5. 消融实验；
|
||||||
|
6. 压力、故障和拓扑实验；
|
||||||
|
7. 大规模仿真性能；
|
||||||
|
8. 敏感性和不确定性；
|
||||||
|
9. 失败案例、模型边界与改进；
|
||||||
|
10. 结论。
|
||||||
|
|
||||||
|
每项结论采用“实验条件—观察数据—结论—适用范围”的格式。例如，不应只写“指数退避更好”，而应写明在哪些负载、超时率和重试次数下改善了哪些指标，以及是否牺牲了任务完成时间。
|
||||||
|
|
||||||
|
# 结论
|
||||||
|
|
||||||
|
本实验计划通过实现正确性、参数可信度、预测精度、基线对比、消融、压力、故障、拓扑、规模和敏感性十类实验，形成从代码到结论的完整证据链。执行过程中应优先完成三节点对账和小规模预测，再逐步扩展到高负载及大规模场景；所有结果必须保留参数来源、随机种子和复现配置，确保最终参赛报告可信、透明且可重复。
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
param(
|
||||||
|
[Parameter(Mandatory=$true)][string]$InputDirectory,
|
||||||
|
[Parameter(Mandatory=$true)][string]$PdfDirectory
|
||||||
|
)
|
||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
New-Item -ItemType Directory -Force -Path $PdfDirectory | Out-Null
|
||||||
|
$word = New-Object -ComObject Word.Application
|
||||||
|
$word.Visible = $false
|
||||||
|
$word.DisplayAlerts = 0
|
||||||
|
try {
|
||||||
|
foreach ($file in Get-ChildItem -LiteralPath $InputDirectory -Filter '*.docx') {
|
||||||
|
$doc = $word.Documents.Open($file.FullName, $false, $true)
|
||||||
|
try {
|
||||||
|
$doc.Repaginate()
|
||||||
|
$pdf = Join-Path $PdfDirectory ($file.BaseName + '.pdf')
|
||||||
|
$doc.ExportAsFixedFormat($pdf, 17)
|
||||||
|
} finally {
|
||||||
|
$doc.Close($false)
|
||||||
|
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($doc) | Out-Null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
$word.Quit()
|
||||||
|
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($word) | Out-Null
|
||||||
|
[GC]::Collect()
|
||||||
|
[GC]::WaitForPendingFinalizers()
|
||||||
|
}
|
||||||
@@ -0,0 +1,158 @@
|
|||||||
|
param(
|
||||||
|
[Parameter(Mandatory=$true)][string]$InputDirectory,
|
||||||
|
[Parameter(Mandatory=$true)][string]$PdfDirectory
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = 'Stop'
|
||||||
|
New-Item -ItemType Directory -Force -Path $PdfDirectory | Out-Null
|
||||||
|
|
||||||
|
$wdAlignParagraphCenter = 1
|
||||||
|
$wdAlignParagraphRight = 2
|
||||||
|
$wdAlignParagraphJustify = 3
|
||||||
|
$wdLineSpaceMultiple = 5
|
||||||
|
$wdPageBreak = 7
|
||||||
|
$wdFormatPDF = 17
|
||||||
|
$wdFieldPage = 33
|
||||||
|
$wdHeaderFooterPrimary = 1
|
||||||
|
$wdCollapseEnd = 0
|
||||||
|
$wdColorWhite = 16777215
|
||||||
|
$wdColorDarkBlue = 9655585
|
||||||
|
$wdColorBlue = 11621185
|
||||||
|
$wdColorGray = 8421504
|
||||||
|
|
||||||
|
$word = New-Object -ComObject Word.Application
|
||||||
|
$word.Visible = $false
|
||||||
|
$word.DisplayAlerts = 0
|
||||||
|
|
||||||
|
try {
|
||||||
|
Get-ChildItem -LiteralPath $InputDirectory -Filter '*.docx' | ForEach-Object {
|
||||||
|
$doc = $word.Documents.Open($_.FullName)
|
||||||
|
try {
|
||||||
|
$doc.PageSetup.PaperSize = 2 # Letter
|
||||||
|
$doc.PageSetup.TopMargin = $word.InchesToPoints(0.82)
|
||||||
|
$doc.PageSetup.BottomMargin = $word.InchesToPoints(0.78)
|
||||||
|
$doc.PageSetup.LeftMargin = $word.InchesToPoints(0.9)
|
||||||
|
$doc.PageSetup.RightMargin = $word.InchesToPoints(0.9)
|
||||||
|
$doc.PageSetup.HeaderDistance = $word.InchesToPoints(0.35)
|
||||||
|
$doc.PageSetup.FooterDistance = $word.InchesToPoints(0.35)
|
||||||
|
|
||||||
|
$normal = $doc.Styles.Item(-1)
|
||||||
|
$normal.Font.Name = 'Calibri'
|
||||||
|
$normal.Font.NameFarEast = 'Microsoft YaHei'
|
||||||
|
$normal.Font.Size = 10.5
|
||||||
|
$normal.Font.Color = 0
|
||||||
|
$normal.ParagraphFormat.Alignment = $wdAlignParagraphJustify
|
||||||
|
$normal.ParagraphFormat.SpaceBefore = 0
|
||||||
|
$normal.ParagraphFormat.SpaceAfter = 6
|
||||||
|
$normal.ParagraphFormat.LineSpacingRule = $wdLineSpaceMultiple
|
||||||
|
$normal.ParagraphFormat.LineSpacing = 15.5
|
||||||
|
|
||||||
|
$title = $doc.Styles.Item(-63)
|
||||||
|
$title.Font.Name = 'Calibri'
|
||||||
|
$title.Font.NameFarEast = 'Microsoft YaHei'
|
||||||
|
$title.Font.Size = 25
|
||||||
|
$title.Font.Bold = $true
|
||||||
|
$title.Font.Color = $wdColorDarkBlue
|
||||||
|
$title.ParagraphFormat.Alignment = $wdAlignParagraphCenter
|
||||||
|
$title.ParagraphFormat.SpaceBefore = 115
|
||||||
|
$title.ParagraphFormat.SpaceAfter = 10
|
||||||
|
$title.ParagraphFormat.KeepWithNext = $true
|
||||||
|
|
||||||
|
$subtitle = $doc.Styles.Item(-75)
|
||||||
|
$subtitle.Font.NameFarEast = 'Microsoft YaHei'
|
||||||
|
$subtitle.Font.Name = 'Calibri'
|
||||||
|
$subtitle.Font.Size = 14
|
||||||
|
$subtitle.Font.Color = $wdColorGray
|
||||||
|
$subtitle.ParagraphFormat.Alignment = $wdAlignParagraphCenter
|
||||||
|
$subtitle.ParagraphFormat.SpaceAfter = 30
|
||||||
|
|
||||||
|
$headingSettings = @(
|
||||||
|
@{ Id=-2; Size=16; Before=16; After=8; Color=$wdColorBlue },
|
||||||
|
@{ Id=-3; Size=13; Before=12; After=6; Color=$wdColorBlue },
|
||||||
|
@{ Id=-4; Size=11.5; Before=9; After=4; Color=$wdColorDarkBlue }
|
||||||
|
)
|
||||||
|
foreach ($h in $headingSettings) {
|
||||||
|
$style = $doc.Styles.Item($h.Id)
|
||||||
|
$style.Font.Name = 'Calibri'
|
||||||
|
$style.Font.NameFarEast = 'Microsoft YaHei'
|
||||||
|
$style.Font.Size = $h.Size
|
||||||
|
$style.Font.Bold = $true
|
||||||
|
$style.Font.Color = $h.Color
|
||||||
|
$style.ParagraphFormat.SpaceBefore = $h.Before
|
||||||
|
$style.ParagraphFormat.SpaceAfter = $h.After
|
||||||
|
$style.ParagraphFormat.KeepWithNext = $true
|
||||||
|
$style.ParagraphFormat.KeepTogether = $true
|
||||||
|
$style.ParagraphFormat.PageBreakBefore = $false
|
||||||
|
}
|
||||||
|
|
||||||
|
foreach ($p in $doc.Paragraphs) {
|
||||||
|
$p.Range.Font.NameFarEast = 'Microsoft YaHei'
|
||||||
|
if ($p.Range.Text.Trim() -eq '目录') {
|
||||||
|
$p.Alignment = $wdAlignParagraphCenter
|
||||||
|
$p.Range.Font.Size = 18
|
||||||
|
$p.Range.Font.Bold = $true
|
||||||
|
$p.Range.Font.Color = $wdColorDarkBlue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
foreach ($table in $doc.Tables) {
|
||||||
|
$table.AllowAutoFit = $true
|
||||||
|
$table.AutoFitBehavior(2)
|
||||||
|
$table.Rows.AllowBreakAcrossPages = $true
|
||||||
|
$table.Range.Font.Name = 'Calibri'
|
||||||
|
$table.Range.Font.NameFarEast = 'Microsoft YaHei'
|
||||||
|
$table.Range.Font.Size = 9
|
||||||
|
$table.Range.ParagraphFormat.SpaceAfter = 2
|
||||||
|
$table.Range.ParagraphFormat.LineSpacingRule = 0
|
||||||
|
$table.Borders.Enable = 1
|
||||||
|
if ($table.Rows.Count -gt 0) {
|
||||||
|
$table.Rows.Item(1).Range.Font.Bold = $true
|
||||||
|
$table.Rows.Item(1).Range.Font.Color = $wdColorWhite
|
||||||
|
$table.Rows.Item(1).Shading.BackgroundPatternColor = $wdColorDarkBlue
|
||||||
|
$table.Rows.Item(1).HeadingFormat = $true
|
||||||
|
}
|
||||||
|
$table.Range.Cells.VerticalAlignment = 1
|
||||||
|
}
|
||||||
|
|
||||||
|
foreach ($section in $doc.Sections) {
|
||||||
|
$header = $section.Headers.Item($wdHeaderFooterPrimary)
|
||||||
|
$header.Range.Text = '大规模多智能体网络流量建模与预测'
|
||||||
|
$header.Range.Font.NameFarEast = 'Microsoft YaHei'
|
||||||
|
$header.Range.Font.Name = 'Calibri'
|
||||||
|
$header.Range.Font.Size = 8.5
|
||||||
|
$header.Range.Font.Color = $wdColorGray
|
||||||
|
$header.Range.ParagraphFormat.Alignment = 0
|
||||||
|
|
||||||
|
$footer = $section.Footers.Item($wdHeaderFooterPrimary)
|
||||||
|
$footer.Range.Text = '参赛材料初稿 | '
|
||||||
|
$footer.Range.Font.NameFarEast = 'Microsoft YaHei'
|
||||||
|
$footer.Range.Font.Name = 'Calibri'
|
||||||
|
$footer.Range.Font.Size = 8.5
|
||||||
|
$footer.Range.Font.Color = $wdColorGray
|
||||||
|
$footer.Range.ParagraphFormat.Alignment = $wdAlignParagraphRight
|
||||||
|
$range = $footer.Range
|
||||||
|
$range.Collapse($wdCollapseEnd)
|
||||||
|
[void]$footer.Range.Fields.Add($range, $wdFieldPage)
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($doc.TablesOfContents.Count -gt 0) {
|
||||||
|
$doc.TablesOfContents.Item(1).Update()
|
||||||
|
}
|
||||||
|
$doc.Fields.Update() | Out-Null
|
||||||
|
$doc.Repaginate()
|
||||||
|
$doc.Save()
|
||||||
|
$pdfPath = Join-Path $PdfDirectory ($_.BaseName + '.pdf')
|
||||||
|
$doc.ExportAsFixedFormat($pdfPath, $wdFormatPDF)
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
$doc.Close($true)
|
||||||
|
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($doc) | Out-Null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
$word.Quit()
|
||||||
|
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($word) | Out-Null
|
||||||
|
[GC]::Collect()
|
||||||
|
[GC]::WaitForPendingFinalizers()
|
||||||
|
}
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
param([Parameter(Mandatory=$true)][string]$Directory)
|
||||||
|
$ErrorActionPreference='Stop'
|
||||||
|
Add-Type -AssemblyName System.IO.Compression.FileSystem
|
||||||
|
$ns='http://schemas.openxmlformats.org/wordprocessingml/2006/main'
|
||||||
|
|
||||||
|
function Set-Attr($node,$name,$value,$xml){
|
||||||
|
$attr=$xml.CreateAttribute('w',$name,$ns); $attr.Value=[string]$value
|
||||||
|
[void]$node.Attributes.SetNamedItem($attr)
|
||||||
|
}
|
||||||
|
function Ensure-Child($parent,$local,$xml){
|
||||||
|
$child=$parent.SelectSingleNode("w:$local",$script:nsmgr)
|
||||||
|
if(-not $child){$child=$xml.CreateElement('w',$local,$ns); [void]$parent.AppendChild($child)}
|
||||||
|
return $child
|
||||||
|
}
|
||||||
|
function Set-Style($xml,$styleId,$font,$eastAsia,$sizeHalf,$color,$bold,$before,$after,$line){
|
||||||
|
$s=$xml.SelectSingleNode("//w:style[@w:styleId='$styleId']",$script:nsmgr); if(-not $s){return}
|
||||||
|
$rPr=Ensure-Child $s 'rPr' $xml
|
||||||
|
$fonts=Ensure-Child $rPr 'rFonts' $xml; Set-Attr $fonts 'ascii' $font $xml; Set-Attr $fonts 'hAnsi' $font $xml; Set-Attr $fonts 'eastAsia' $eastAsia $xml
|
||||||
|
$sz=Ensure-Child $rPr 'sz' $xml; Set-Attr $sz 'val' $sizeHalf $xml
|
||||||
|
$szCs=Ensure-Child $rPr 'szCs' $xml; Set-Attr $szCs 'val' $sizeHalf $xml
|
||||||
|
$c=Ensure-Child $rPr 'color' $xml; Set-Attr $c 'val' $color $xml
|
||||||
|
$b=$rPr.SelectSingleNode('w:b',$script:nsmgr)
|
||||||
|
if($bold -and -not $b){$b=$xml.CreateElement('w','b',$ns); [void]$rPr.AppendChild($b)} elseif(-not $bold -and $b){[void]$rPr.RemoveChild($b)}
|
||||||
|
$pPr=Ensure-Child $s 'pPr' $xml
|
||||||
|
$spacing=Ensure-Child $pPr 'spacing' $xml; Set-Attr $spacing 'before' $before $xml; Set-Attr $spacing 'after' $after $xml; Set-Attr $spacing 'line' $line $xml; Set-Attr $spacing 'lineRule' 'auto' $xml
|
||||||
|
}
|
||||||
|
|
||||||
|
foreach($file in Get-ChildItem -LiteralPath $Directory -Filter '*.docx'){
|
||||||
|
$tmp=Join-Path $Directory ('.tmp_'+[guid]::NewGuid().ToString('N'))
|
||||||
|
[IO.Compression.ZipFile]::ExtractToDirectory($file.FullName,$tmp)
|
||||||
|
try{
|
||||||
|
[xml]$styles=Get-Content -Raw -Encoding UTF8 (Join-Path $tmp 'word\styles.xml')
|
||||||
|
$script:nsmgr=New-Object Xml.XmlNamespaceManager($styles.NameTable); $nsmgr.AddNamespace('w',$ns)
|
||||||
|
Set-Style $styles 'Normal' 'Calibri' 'Microsoft YaHei' 21 '000000' $false 0 120 300
|
||||||
|
Set-Style $styles 'Title' 'Calibri' 'Microsoft YaHei' 50 '17365D' $true 1200 240 240
|
||||||
|
Set-Style $styles 'Subtitle' 'Calibri' 'Microsoft YaHei' 28 '666666' $false 0 360 280
|
||||||
|
Set-Style $styles 'Heading1' 'Calibri' 'Microsoft YaHei' 32 '2E74B5' $true 320 160 280
|
||||||
|
Set-Style $styles 'Heading2' 'Calibri' 'Microsoft YaHei' 26 '2E74B5' $true 240 120 280
|
||||||
|
Set-Style $styles 'Heading3' 'Calibri' 'Microsoft YaHei' 23 '1F4D78' $true 180 80 280
|
||||||
|
$styles.Save((Join-Path $tmp 'word\styles.xml'))
|
||||||
|
|
||||||
|
[xml]$doc=Get-Content -Raw -Encoding UTF8 (Join-Path $tmp 'word\document.xml')
|
||||||
|
$script:nsmgr=New-Object Xml.XmlNamespaceManager($doc.NameTable); $nsmgr.AddNamespace('w',$ns)
|
||||||
|
foreach($sect in $doc.SelectNodes('//w:sectPr',$nsmgr)){
|
||||||
|
$pgSz=Ensure-Child $sect 'pgSz' $doc; Set-Attr $pgSz 'w' 12240 $doc; Set-Attr $pgSz 'h' 15840 $doc
|
||||||
|
$pgMar=Ensure-Child $sect 'pgMar' $doc; Set-Attr $pgMar 'top' 1224 $doc; Set-Attr $pgMar 'right' 1296 $doc; Set-Attr $pgMar 'bottom' 1152 $doc; Set-Attr $pgMar 'left' 1296 $doc; Set-Attr $pgMar 'header' 504 $doc; Set-Attr $pgMar 'footer' 504 $doc; Set-Attr $pgMar 'gutter' 0 $doc
|
||||||
|
}
|
||||||
|
foreach($tbl in $doc.SelectNodes('//w:tbl',$nsmgr)){
|
||||||
|
$tblPr=Ensure-Child $tbl 'tblPr' $doc
|
||||||
|
$style=Ensure-Child $tblPr 'tblStyle' $doc; Set-Attr $style 'val' 'TableGrid' $doc
|
||||||
|
$layout=Ensure-Child $tblPr 'tblLayout' $doc; Set-Attr $layout 'type' 'autofit' $doc
|
||||||
|
$first=$tbl.SelectSingleNode('w:tr[1]',$nsmgr)
|
||||||
|
if($first){
|
||||||
|
$trPr=Ensure-Child $first 'trPr' $doc; $hdr=Ensure-Child $trPr 'tblHeader' $doc; Set-Attr $hdr 'val' 1 $doc
|
||||||
|
foreach($cell in $first.SelectNodes('w:tc',$nsmgr)){
|
||||||
|
$tcPr=Ensure-Child $cell 'tcPr' $doc; $shd=Ensure-Child $tcPr 'shd' $doc; Set-Attr $shd 'fill' '17365D' $doc
|
||||||
|
foreach($rPr in $cell.SelectNodes('.//w:rPr',$nsmgr)){ $c=Ensure-Child $rPr 'color' $doc; Set-Attr $c 'val' 'FFFFFF' $doc; if(-not $rPr.SelectSingleNode('w:b',$nsmgr)){[void]$rPr.AppendChild($doc.CreateElement('w','b',$ns))} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$doc.Save((Join-Path $tmp 'word\document.xml'))
|
||||||
|
|
||||||
|
foreach($hf in Get-ChildItem -LiteralPath (Join-Path $tmp 'word') -Filter 'header*.xml' -ErrorAction SilentlyContinue){
|
||||||
|
[xml]$hx=Get-Content -Raw -Encoding UTF8 $hf.FullName
|
||||||
|
$script:nsmgr=New-Object Xml.XmlNamespaceManager($hx.NameTable); $nsmgr.AddNamespace('w',$ns)
|
||||||
|
foreach($t in $hx.SelectNodes('//w:t',$nsmgr)){$t.InnerText='大规模多智能体网络流量建模与预测'}
|
||||||
|
foreach($rPr in $hx.SelectNodes('//w:rPr',$nsmgr)){
|
||||||
|
$fonts=Ensure-Child $rPr 'rFonts' $hx; Set-Attr $fonts 'eastAsia' 'Microsoft YaHei' $hx
|
||||||
|
$sz=Ensure-Child $rPr 'sz' $hx; Set-Attr $sz 'val' 17 $hx
|
||||||
|
$c=Ensure-Child $rPr 'color' $hx; Set-Attr $c 'val' '777777' $hx
|
||||||
|
}
|
||||||
|
$hx.Save($hf.FullName)
|
||||||
|
}
|
||||||
|
|
||||||
|
$new=$file.FullName+'.new'
|
||||||
|
if(Test-Path $new){Remove-Item -LiteralPath $new}
|
||||||
|
[IO.Compression.ZipFile]::CreateFromDirectory($tmp,$new)
|
||||||
|
Copy-Item -Force -LiteralPath $new -Destination $file.FullName
|
||||||
|
Remove-Item -LiteralPath $new
|
||||||
|
} finally { Remove-Item -Recurse -Force -LiteralPath $tmp }
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user