Review Draft

Loop Engineering:不是手写 Prompt,而是用循环开发agent

Loop Engineering: Not Hand-Writing Prompts, but Developing Agents Through Loops

Prompt 仍然重要,但它不应该停留在一次性手写指令。真正的变化,是把测试、审核、失败日志和真实结果反馈放进 Skill 的开发循环。

Prompts still matter, but they should not remain one-off instructions written by hand. The real change is putting tests, reviews, failure logs, and real output feedback into the development loop of a Skill.

一、这句话到底该怎么理解

最近有一句话很火:

You shouldn't be prompting coding agents anymore. You should be designing loops that prompt your agents.

我更愿意把它理解为:

你不应该再靠一条条手写提示词去操作 coding agent。你应该设计循环,让循环去生成、约束、检查并修正 agent 需要的提示词和任务。

这里的重点不是“不要写 prompt”。相反,prompt 仍然是 agent 工作的核心输入。真正变化的是:prompt 不再是人每次临场手写的一句话,而是由一套可验证的循环持续修正出来的工作流产物。

二、最贴近的例子不是普通应用,而是 Skill 开发

如果只是做一个普通小工具,Loop Engineering 可能听起来比较抽象。但如果你开发过 agent Skill,就会发现这个概念非常直接。

因为 Skill 本质上就是把一组 prompt、规则、参考材料、输入输出约束和执行步骤封装起来,让 agent 在特定任务里稳定工作。

所以,Skill 开发最容易体现这句话的含义:

不应该用一次性 prompt 的方式开发 Skill。要设计一个循环,让测试结果、真实运行失败、人工审核意见不断回流,持续修正 Skill 里的 prompt 和规则。

三、虚构例子:开发一个“会议纪要整理 Skill”

假设你想开发一个 Skill,帮助产品经理整理会议纪要。输入是会议录音转写文本,输出是结构化的会议结论、待办事项、风险和待确认问题。

第一版你可能只写一个 prompt:

请根据会议录音转写内容,整理出会议结论、待办事项、负责人和风险。

这个 prompt 第一次看起来能用,但多跑几次就会暴露问题:

  • 把讨论中的猜测写成正式结论。
  • 给没有明确 owner 的待办事项硬编负责人。
  • 漏掉“暂不处理”“下次再议”这类状态。
  • 输出格式不稳定,有时是表格,有时是段落。
  • 把会议里的闲聊内容也整理进正式纪要。

如果你只是继续补 prompt,比如“请更严谨一点”“不要编造负责人”“请按固定格式输出”,这仍然只是 Prompt Engineering。

四、从 Prompt Engineering 到 Loop Engineering

Skill 开发里的 loop,会把这个任务拆成更稳定的结构:

meeting-notes-skill/
  SKILL.md
  references/
    extraction-rules.md
    output-schema.md
    review-checklist.md
  tests/
    golden/
      owner-pending-input.txt
      owner-pending-expected.json
      decision-vs-discussion-input.txt
      decision-vs-discussion-expected.json

其中 SKILL.md 不只是“请整理会议纪要”。它需要说明 Skill 的目标、输入范围、输出结构和禁止行为。

extraction-rules.md 定义什么能算会议结论,什么只能算待确认事项,什么必须忽略。

output-schema.md 固定输出结构,例如:

{
  "decisions": [],
  "action_items": [],
  "risks": [],
  "open_questions": [],
  "ignored_content": []
}

tests/golden/ 保存黄金测试集:典型输入、期望输出和禁止输出。

五、一个具体失败如何进入循环

假设某次测试输入里有一句话:

张三可能可以跟一下这个事情,不过还没定。

第一版 Skill 输出:

{
  "action_items": [
    {
      "task": "跟进该事项",
      "owner": "张三"
    }
  ]
}

这个结果是错的。因为原文说的是“可能”“还没定”,负责人并未确认。

如果只是临时改 prompt,你可能会写:

不要把不确定的负责人写成确定负责人。

但 Loop Engineering 的做法,是把这次失败沉淀成规则和测试。

新增规则

如果 owner 前面或后面出现“可能”“也许”“暂定”“还没定”“待确认”等不确定表达,
不能写入 confirmed owner。
只能写入 open_questions 或 action_items.pending_owner。

新增黄金测试

输入:
“张三可能可以跟一下这个事情,不过还没定。”

期望:
open_questions 包含“负责人待确认”。

禁止:
action_items.owner = "张三"

这样下一次你修改 Skill,不管是改 prompt、改参考规则,还是改输出结构,都必须跑这个测试。只要它又把张三写成确定负责人,就说明 Skill 退化了。

关键点:这不是让 agent 多跑几次,而是让每一次错误都改变 Skill 的下一轮行为。

六、Loop 的发动机不是 prompt,而是验证反馈

很多人讲 Loop Engineering,只讲“循环调用 agent”。但没有验证器的 loop,只是重复 prompt。有验证器的 loop,才是工程系统。

Skill 开发里的验证反馈通常有三类。

1. 黄金测试集

把典型输入和期望输出固定下来。每次改 Skill,都跑一遍。如果以前能处理的案例现在坏了,就说明这次修改引入了退化。

2. 确定性校验

能用代码判断的事情,不要交给 prompt。例如:

  • JSON schema 是否符合。
  • 必填字段是否存在。
  • 输出章节是否完整。
  • 禁止表达是否出现。
  • 待确认事项是否被误写成已确认结论。
  • 失败日志是否写入。

3. 真实结果反馈

真实会议材料里发现的新错误,不应该只停留在聊天记录里。它应该脱敏后进入测试集,或者变成新的规则、schema、review checklist。

这样系统才会真的变聪明。否则你只是每次重新提醒 agent,不是在开发 Skill。

七、一套 Skill 开发 Loop 可以长这样

写第一版 Skill prompt
-> 准备黄金测试集
-> 运行 Skill
-> 检查 schema 和固定规则
-> 人工审核输出
-> 归因失败:prompt 模糊、规则缺失、schema 不严、测试不足
-> 修改 SKILL.md / references / schema / tests
-> 用真实材料试运行
-> 脱敏真实失败案例并加入回归测试
-> 下一轮

这就是 “designing loops that prompt your agents”。不是人每次临场提醒 agent,而是 Skill 开发循环不断改进下一轮 agent 会收到的提示词。

八、这句话的准确含义

所以,这句话不应该被理解成“以后不用 prompt 了”。更准确的理解是:

你不应该用一次性 prompt 的方式开发 agent 能力。你应该设计一个反馈循环,让测试、审核、真实失败和运行结果持续修正 prompt、规则、schema 和 Skill。

Prompt 是输入。Skill 是封装。Loop 是进化机制。

真正的分水岭不是会不会写一句漂亮 prompt,而是能不能让系统记住失败,并在下一轮自动避免同样的失败。

一句话总结:Prompt Engineering 解决的是“这一次怎么说”。Loop Engineering 解决的是“系统下一次如何变得更可靠”。

1. How this sentence should really be read

One line has been circulating a lot lately:

You shouldn't be prompting coding agents anymore. You should be designing loops that prompt your agents.

I prefer to read it like this:

You should stop driving coding agents by writing prompts one at a time. You should design loops, and let those loops generate, constrain, inspect, and correct the prompts and tasks the agent needs.

The emphasis is not "stop writing prompts". The prompt is still the core input to an agent's work. What has actually changed is this: the prompt is no longer a sentence a person dashes off on the spot each time, but a workflow artifact that a verifiable loop keeps correcting.

2. The closest example is not an ordinary app but Skill development

If all you are building is a small utility, Loop Engineering can sound abstract. But if you have built an agent Skill, the idea becomes immediate.

A Skill is essentially an encapsulated bundle — prompts, rules, reference material, input and output constraints, execution steps — that lets an agent work steadily on a particular task.

Which is why Skill development shows this sentence most plainly:

A Skill should not be developed as a pile of one-off prompts. Design a loop instead, so that test results, real runtime failures, and human review comments keep feeding back in, continuously correcting the prompts and rules inside the Skill.

3. A made-up example: building a "meeting-notes Skill"

Suppose you want a Skill that helps product managers organize meeting notes. The input is the transcript of a recorded meeting; the output is structured meeting decisions, action items, risks, and open questions.

Your first version may be nothing more than a single prompt:

Please read the meeting transcript and organize it into meeting decisions, action items, owners, and risks.

It looks serviceable on the first run, but a few runs later the problems surface:

  • Speculation from the discussion gets written up as formal conclusions.
  • Action items with no clear owner are given owners anyway.
  • States like "not handling this yet" and "revisit next time" get dropped.
  • The output format swings — a table one moment, paragraphs the next.
  • Casual chatter from the meeting ends up in the formal minutes.

If you respond by only bolting more instructions onto the prompt — "be more rigorous", "do not invent owners", "output a fixed format" — you are still doing Prompt Engineering.

4. From Prompt Engineering to Loop Engineering

In Skill development, the loop breaks this task into a more stable structure:

meeting-notes-skill/
  SKILL.md
  references/
    extraction-rules.md
    output-schema.md
    review-checklist.md
  tests/
    golden/
      owner-pending-input.txt
      owner-pending-expected.json
      decision-vs-discussion-input.txt
      decision-vs-discussion-expected.json

SKILL.md is more than "please organize the meeting notes". It has to spell out the Skill's goal, its input scope, its output structure, and the behaviour it must not exhibit.

extraction-rules.md defines what may count as a meeting decision, what may only count as an open question, and what must be discarded.

output-schema.md fixes the output structure, for example:

{
  "decisions": [],
  "action_items": [],
  "risks": [],
  "open_questions": [],
  "ignored_content": []
}

tests/golden/ holds the golden test set: typical inputs, expected outputs, and forbidden outputs.

5. How one concrete failure joins the loop

Suppose this sentence appears in one of your test inputs:

Zhang San may be able to pick this up, but nothing is settled yet.

The first version of the Skill answers:

{
  "action_items": [
    {
      "task": "Pick this item up",
      "owner": "Zhang San"
    }
  ]
}

The answer is wrong: the original says "may" and "nothing is settled yet", so the owner is not confirmed.

Had you only patched the prompt on the spot, you might have written:

Do not record an unconfirmed owner as a confirmed owner.

Loop Engineering instead converts this failure into a rule and a test.

New rule

If words of uncertainty — "may", "maybe", "tentative", "not settled yet", "to be confirmed" — appear before or after an owner,
that person must not be recorded as a confirmed owner.
The item may only be written into open_questions or action_items.pending_owner.

New golden test

Input:
"Zhang San may be able to pick this up, but nothing is settled yet."

Expected:
open_questions contains "owner to be confirmed".

Forbidden:
action_items.owner = "Zhang San"

From now on, whenever you change the Skill — prompt, reference rules, or output structure alike — this test must run. If it once again records Zhang San as the confirmed owner, the Skill has regressed.

Key point: this is not about running the agent a few more times; it is about every error changing what the Skill does next round.

6. The loop runs on validation feedback, not on prompts

Most people who talk about Loop Engineering talk only about "calling the agent in a loop". A loop without a validator is merely prompting in circles. A loop with a validator is an engineering system.

Validation feedback in Skill development generally comes in three kinds.

1. Golden test set

Pin down the typical inputs and their expected outputs, and run them every time the Skill changes. A case that used to pass and now fails means the change introduced a regression.

2. Deterministic checks

Whatever code can decide should never be left to the prompt. For example:

  • Whether the JSON schema is valid.
  • Whether required fields are present.
  • Whether every output section is complete.
  • Whether any forbidden expression appears.
  • Whether an open question has been written up as a confirmed conclusion.
  • Whether the failure log was written.

3. Real result feedback

New errors found in real meeting material should not stay buried in a chat log. Desensitized, they should join the test set, or turn into a new rule, a schema, or a review checklist item.

Only then does the system genuinely get smarter. Otherwise you are simply re-briefing the agent from scratch every time — that is not developing a Skill.

7. A Skill development loop might look like this

Write the first version of the Skill prompt
-> Prepare the golden test set
-> Run the Skill
-> Check the schema and the fixed rules
-> Review the output by hand
-> Attribute the failure: vague prompt, missing rule, loose schema, weak tests
-> Amend SKILL.md / references / schema / tests
-> Trial run with real material
-> Desensitize the real failure and add it to the regression tests
-> Next round

This is what "designing loops that prompt your agents" means. Not a person reminding the agent afresh each time, but a Skill development loop that keeps improving the prompts the agent will receive next round.

8. The precise meaning of the sentence

So this sentence should not be taken to mean "prompts will not be needed from now on". The more accurate reading is:

You should not build agent capabilities out of one-off prompts. You should design a feedback loop in which tests, reviews, real failures, and runtime results keep refining the prompt, the rules, the schema, and the Skill.

The prompt is the input. The Skill is the packaging. The loop is the mechanism of evolution.

The real watershed is not whether you can write a beautiful prompt; it is whether you can make the system remember its failures and avoid them automatically next round.

One-sentence summary: Prompt Engineering answers "how do I say it this time?"; Loop Engineering answers "how does the system become more reliable next time?"