Skip to main content

A board of agents or one agent in a loop: Hermes Agent, part 3

A short summary of the experiment: what I expected from the board, what happened, and what other teams report

Updated 6 min readMarkdown
Part of seriesA multi-agent build with Hermes Agent

This is a short summary of the experiment I described in two posts. In part 1 a team of AI agents on a Hermes Agent Kanban board built one level of my browser game: 38 hours, 22 of them with no agent running. In part 2 I read that board as a distributed system and went through five of its failures.

The question left is simple. Did the multi-agent setup give me anything over the way I usually work?

What I compare it with

I have a lot of experience with simple harnesses: Claude Code, Codex, OpenCode. They all work as one agent in a loop. One main agent holds the task, does the work or hands a small, concrete piece to a cheaper agent, reads the result and decides what comes next. All the coordination stays with that agent, and it uses the harness's own tools for it, which work well.

I did not build the same level a second time that way, so this is not a measured comparison. It is what I saw in this run, next to what I know from everyday work.

What I expected from the board, and what happened

Autonomy. I expected the board to run on its own, with a strong model stepping in now and then. It did not. A coordinating agent outside the board kept it moving the whole time: it added 44 of the 68 cards, restored a lost handoff by hand, did the integrations only it was allowed to do, and kept the board's own machinery working. It cost about $350 in tokens at list prices, against about $40 for all the agents on the board.

Parallel work. I expected several agents to work at once. Most of the time there was one agent or none. For 22 of the 38 hours nobody was running, waiting on limits that did not fit the work, handoffs that got lost and alerts that went to the wrong place.

Cheaper work. The work itself did get cheap: the board's agents together cost about $40. But the judgement did not move off the strong model, and that was most of the bill.

So in this run, because of the mistakes I described in the first two parts, I did not see a real advantage of the multi-agent setup over a plain loop. Autonomy did not happen, and parallel work did not happen either.

What I would keep

The board is a good place for knowledge and history. Every card, run and decision stays in one database, and I use boards like this in real work for exactly that.

What I doubt is the board as the thing agents work from while they implement. People and agents may need different views of the same work: a board for people, and for agents something closer to a graph of tasks with explicit dependencies. Used during implementation as it is, the board either does not fit the agents or costs extra setup and management.

Where a simple loop falls short

Simple loops are hard to run in parallel without some layer above them. That is my opinion, and it can be checked. For one to three tasks at a time, each in its own session, a simple loop works well for me. Beyond that, something has to hand out the work and keep it consistent.

Others ran into the same problems

After the board I read how other teams run several agents at once. Where agents really coordinate with each other, the problems I hit on a small task show up in their reports too.

Cursor ran hundreds of agents on one project for weeks. At first agents claimed tasks from a shared file with locks. They held the locks too long, and "twenty agents would slow down to the effective throughput of two or three, with most time spent waiting." A separate integrator role "created more bottlenecks than it solved", and they removed it. On my board only the coordinator was allowed to integrate, and for 22 of 38 hours nobody was running.

Anthropic, describing its multi-agent research system, says that "LLM agents are not yet great at coordinating and delegating to other agents in real time", and that "most coding tasks involve fewer truly parallelizable tasks than research". Their system uses about 15 times the tokens of a chat. On my board the coordinator cost about $350, against about $40 for all the agents doing the work.

The setups that work look like what works for me: one agent in a loop that hands out work, or many agents that never talk to each other. Zach Wills runs subagents every day, across several projects at once. In each project one long-lived main session holds the plan, the decisions and the integration; it sends subagents off with a task and gets a report back. The subagents do not talk to each other, and two tasks that need the same file run one after the other. I would not call this multi-agent development: it is a plain loop with delegation, grown to a larger scale, and he steers the main sessions himself. At Stripe more than a thousand pull requests a week come from agents that each take one task in their own sandbox and do not coordinate with each other.

Where many agents did coordinate and it worked, the coordination went through git and tests. In Anthropic's C compiler experiment 16 agents shared one repository with no coordinator: they claimed tasks with lock files that git kept consistent, and the tests caught what they broke. The author spent most of his effort on the tests and the environment around the agents.

So my board did not fail in some unusual way. These are the known problems of coordinating agents. The teams that get past them first invest a lot in tests and in the environment. On a small task that investment does not pay off, and the cost of coordination is all you see.

This is not a verdict on Hermes

I do not want to say that Hermes is bad. This particular attempt did not work out, mostly because of how I set it up. After the board I tried Hermes again, with a task graph instead of a board, and that is a different story for another post.

References

No comments yet. Be the first to comment.

A multi-agent build with Hermes Agent

Part 3 of 3

One level of a browser game, built by a team of AI agents on a Hermes Agent Kanban board. Part 1: what happened in 38 hours and what it cost. Part 2: the same run read as a distributed system, failure by failure.

Start from the beginning48 closed cards, no passing design: a multi-agent build with Hermes Agent, part 1What a Kanban board of AI agents built in 38 hours, where the time went and what it cost