The Beef and the Boilerplate

Before you begin

Now let's talk about how to actually do it.

A typical starting point looks something like this. New project, new team, half the people have never used AI tools beyond ChatGPT, and the other half have used something different, individually, on a different stack. Everybody's eager. Licences are in, agents are running, and the first week produces an impressive amount of code. Looks great in the demo. Then week two hits and you notice: three different error handling patterns, two competing folder structures, tests that mock things in ways nobody agreed on, and a merge conflict rate that makes you question your life choices. The tools worked fine. The problem was that nobody had defined what "fine" means for this project. On top of that, the inherent randomness of outputs means that even with the same instructions, you get different results every time, and the sheer mass of generated code to verify is just too much too quickly. Getting to factory-level consistency takes weeks at minimum, often more.

What I'm going to cover next is:

A typical start setting you're likely to encounter is:

The beef and the boilerplate

In order to figure out the correct level of abstraction for your specifications, you need to understand the fundamental distinction between two categories of work in AI-assisted delivery: the "beef" that requires real craft, and the "boilerplate" where agents earn their keep.

Next, I'll explain why this matters.

The beef: Work that requires your craft

The beef is the activities where your judgment is irreplaceable. This is where you say what to build.

For the beef to be actually digestible, let's say medium, we need to ensure that the following are in place:

Acceptance criteria. The Pareto principle is very much alive in the agentic world as well. Much of the error often goes to the edge cases, however improbable they might be. Vagueness does not help here either: spend time, e.g., by using Gherkin format to define the paths that need to be covered, and ask AI to discover more. "It should work correctly" is not a criterion.

Other talking points you might have come across are that AI is generating technical debt at scale and introducing subtle bugs that are hard to detect. Both of these, among others, are valid concerns and to be taken seriously: it's a whole other ballgame to spend weeks refactoring things in the middle of the project or starting over, especially after you've already shipped something, with AI or not. Don't trust you can just "make the things right later". You really can't.

Skip the boilerplate: Where agents earn their keep

The boilerplate refers to things that should be rather straightforward. It consists of activities where agents reliably produce acceptable output. So 'boilerplate' basically refers to making the boring part automatic.

Examples of these kinds of activities include:

The guardrails for delegated work

An essential safety mechanism for agents to perform is to erect boundaries and hard guardrails.

On the top level, these fences are baked-in state tracking which requires certain tasks, states, and viewpoints to have been considered, and isolating the unit of work with branching and PRs from the already verified content.

Once you have a good agreement and tooling to support that, you should consider the feature-level guidance to keep things on track. Consider the following:

How precise does the spec need to be?

The current generation of developers is not used to specifying work in detail and to be frank, it was not that much better pre-Agile either, or at least it was often different guys who did the plan.

So I've been thinking a lot about what's the suitable level of detail that's both realistic to get and detailed enough for the agents. Also a sanity check needs to be made: a quick bug fix, a small detail added or a small layout adjustment might not need anything on paper.

Chapter 13 made the case for why specification matters and introduced three maturity levels (spec-first, spec-anchored, spec-as-source). Here's the practical side: how much specification effort different work actually needs. Generally speaking, by the "The Calibration Principle" applied to agentic software engineering Specification effort should be proportional to the cost of getting it wrong. I'll postulate the following:

I encourage discussing these things with your team; it's nothing new and something that should be agreed even without AI. Also consider measuring these things like I suggest in Knowing It Works to get some data on the tasks which consistently come back for revision for bugs, missing features or architectural issues.

Antipatterns and bad, bad practices you should avoid

As I mentioned in the "focus on DONT's over DO's" principle in Chapter 9, it's often more instructive to look at what not to do than what to do.

This applies also on pattern level. I collected some well-known antipatterns below from the literature and some I've discovered in my work.

Many of the DONT's and DO's here are essentially an art of prompt engineering applied to the entire delivery process. So the change here is that we feed the prompts through the planning artifacts from files or similar instead of writing them ourselves. In the end, however, anything that you write or synthesize with AI is a prompt for the next step.

Specification anti-patterns

Here's what to say and what not when writing specifications for features or feature-independent general project documentation. Sure, you'll get the idea; the entire point is to be specific and explicit. Once you've set up your Software Factory and all the conventions and patterns properly in place, you can loosen the noose.

Agent design anti-patterns

Another group of anti-patterns I've encountered is about the agents themselves. Remember, agents are still LLM calls with a 'recipe' i.e. the intent, tasks the agent is supposed to do from your execution pipeline, and the context it should operate with (files, category, module). The last one could be already specified during planning, but the agent should be free enough to find more when needed.

To recap, remember that all these 'agents' are is: Agent = Recipe + Context + Tools

This is basically just an md file with thin YAML frontmatter. The 'standard' (note the hyphens) YAML is just a name, tool list, model to use followed by what's basically a prompt to the LLM. All agents should be general enough to perform any task your planning practice produces in the provided context.

Process anti-patterns

If you're serious about the automation, entirely possible to achieve with modern AI tech, you need to be serious also about the process. Here are the pitfalls I see most often.

Security & governance anti-patterns

This category of donts is not only about agentic development, but these apply to all usage of generative AI systems. The more you offload to AI without proper oversight, the more risk surface you expose to having the prod connection strings injected into your codebase or various tool memories.

Building your software factory

So how to get started with the factory? The following sections suggest a plan to follow to build it gradually, applying the same incremental approach from Chapter 8's Adoption Ladder. The point here is not that exactly these seven layers are the gold standard (obviously) but illustrate a logical progression of how I've tried to do this. It's best to build on a strong foundation (solid project context, good documentation structure, task tracking) before jumping to parallel agents runs coding the new major ERP (would be good riddance btw) in days.

All this is obviously going to be iterative. The tools evolve, models and people change, your codebase and documentation will evolve, and your understanding of what belongs in the project context and how to specify work will improve. So in parallel with this regular process you have for your factory, you need to have the maintenance process for the factory.

So what kind of things to look out for and learn about? I tried to summarize this in the next image.

Let's do a quick recap of the concepts here before going into the phases. The core building blocks of your software factory are:

Phase 1: Custom instructions

Start with the "always true" things about your project. A CLAUDE.md, copilot-instructions.md, or equivalent file that captures the conventions, technology choices, and constraints that every agent interaction should respect. Keep this compact. If it grows beyond what fits comfortably in an agent's context window, it's too large. Compare it to your corporate guidebooks: you might have some work place safety rules, stock market rules, GDPR and whatever the European Union has (AI Act, anyone?), but certainly the coding agent won't need to read all of them.

Begin with: brief project intro, perhaps a short vocabulary, technical things like languages, tech stack, documentation locations and such and something generic about the process and architecture. Hard stop things and must-not-do's like "never add any secrets like API keys, connection strings to code" belong here, too.

Phase 2: Hierarchical documentation structure

As the project matures, the single instruction file isn't enough. Or it will be so big that it eats most of your context window; remember, that event basic system prompts, agent recipe, tool specificatons and others are easily tens of thousands of tokens alone.

So, design a documentation hierarchy as suggested in this book: divide guidelines, technology decisions, architectural patterns, domain-specific context, and things like standard UI patterns into individual files. Also describe the process and gating once you're ready with that.

Your document library will grow over time, especially now that you have (over?) eager AI-assistants to add more content, so a good index is essential for your agents to find the right context without reading everything. Structure your library in a way it so agents can be pointed at the relevant subset rather than ingesting the whole tree.

See Chapter 17 for the three-tier documentation model (project-level, task-specific, tracking) and practical techniques for organizing context.

Phase 3: Task tracking and gating design

Introduce structured task tracking that agents can consume and update. I suggest having this in place early on, not because of the 100% complete paper trail, but this is really the key piece to get right in order for anything to work properly.

File-based approaches work well for revision control and collaboration. Key design decisions include what granularity of tasks, what status transitions, what gates between phases you want. Consider what integration with existing tools (Jira, Azure DevOps) is required versus what can live in the repository. Also check what ready-made tools there are: I've mentioned GSD, and also Beads as candidates for this. Or just let the AI generate a tool for you which matches your exact needs.

Phase 4: First agents for selected phases

Don't try to automate every phase at once.

I'd start with planning instead of the coding agent. I know for sure most people think the coding agent is the most tempting. My point here all along was that you'll need this spec-driven thing even if you don't like it, no matter what.

Use models best suited for each agent type (reasoning models for planning, faster or code-oriented models for implementation, UI-focused models for interface design). I've preferred the ever-enthusiastic Sonnet as the orchestrator, GPT Codex for code generation, Gemini for UI-related work, and Opus for planning.

Next, I'd introduce a decent test planner. Start TDD-driven planning early. As with hand crafted code, it's very hard to get good and representative coverage after the fact, especially without proper functional specifications.

Phase 5: Manual gating and review

Phase 6: Automated gating and guardrails

Once you understand the failure modes and potential points where things usually go sideways, you may introduce automated checks. We've done static analysis, architecture rule enforcement, convention validation backed by deterministic tools (like static code analysis) and partially AI capabilities.

Define what manual intervention points are genuinely necessary versus what can be safely automated. I'd keep the requirement to review and approve the plan before implementation.

Phase 7: Skills and tooling

Develop reusable skills and command-line tools for operations that should "always work". These are like basic scaffolding, common transformations, standard validations, compilations, better-than-grep semantic search tools, etc. Chances are you'll find excellent ones online (check for instance skills.sh, awesome skills on GitHub, etc.).

These are important building blocks of the agentic software factory which offload repetitive work from both humans and agents and reduce the surface area where things can go wrong. As a summary, check out the diagram below.

Trust and verification

Governed delivery is not a faith-based initiative. If the process is working, specific numbers improve. If they don't, something is wrong: either with the specs, the gates, the agent configuration, or the team's understanding of the boundary between beef and boilerplate.

Four metrics worth tracking:

Plan rejection rate is your leading indicator. It tells you whether the team is getting better at the beef, the upfront specification work that determines everything downstream. Track it weekly. When it drops, the process is calibrating. When it creeps back up, investigate: new team members? Changed requirements patterns? Specification template drift?

Defect escapes is your lagging indicator. It tells you whether the gates are actually catching what matters. If the same category of defect escapes repeatedly (say, integration issues between services), that's a signal to add a gate check, not to blame the agent.

Summary

When all is set, you'll have covered most of the stuff that is needed to get the most out of your agents and avoid the common pitfalls. It'll look something like this (but not limited to).