Personal experiment / Project 04
CSGClaw: Agent Workflows & Evaluation
Outcome & decision
Compared four agent workflows on one proposal task: more agents did not reliably preserve better evidence. Separately documented 19 product issues for engineering triage, including 12 involving coordination or context isolation.
- My responsibility
- Workflow evaluation, scenario testing, and reproducible issue reporting
- Project scope
- Proposal-workflow experiment; separate OpenCSG dogfooding work
Evidence basis. Task and role prompts, four final reports, retrospective scoring, and an issue export. All 19 documented issues were open in the 4 September 2026 export.
Findings & recommendation
The single-agent reports scored highest in this sample. I recommended explicit evidence handoffs, citation checks, and review of changed claims before accepting a customer-facing proposal.
| Workflow | Retrospective score |
|---|---|
| GPT single agent | 93/100 |
| Domestic single agent | 89/100 |
| Domestic multi-agent | 76/100 |
| Hybrid with GPT reviewer | 69/100 |
Problem & constraints
A platform proposal needs to distinguish verified product capability from proposed architecture. The shared task was an eight-section public-sector LLM and Agent platform proposal, not a deployed customer system.
CSGHub supplied the proposed model, dataset, and AI asset foundation; orchestration and runtime management belonged to a separate architectural layer. Confusing those boundaries made a proposal less dependable.
My work & technical decisions
I designed the task and role prompts, assessed the AI-generated outputs, and traced high-risk claims against available evidence. I evaluated manager, researcher, writer, and reviewer handoffs rather than treating more agents or longer reports as proof of quality.
The recommendation was to share evidence and artifacts explicitly and check what revisions changed. These are proposed improvements; adoption or shipped fixes are not established.
Separate product-validation deliverable
As part of my OpenCSG work, I tested CSGClaw through internal use: multi-room coordination, task-state accuracy, context isolation, shared files, and Windows/Docker behavior. The issue export records 19 reports under my authorship, with reproduction steps and expected versus observed behavior.
The supporting analysis groups six coordination issues and six room/context-isolation issues, totaling 12 of 19. These reports are a technical handoff for engineering triage, not 19 resolved customer incidents.
Limits & interpretation
The comparison used one final output per configuration, retrospective scoring, unequal research and revision effort, and incomplete model settings. Cost, latency, and human effort were not reliably recorded.
The findings support better evidence handling in this task. They do not establish universal model rankings, a causal disadvantage of multi-agent systems, or completed issue fixes.