deliberate.codes

Practical AI for data & software engineers. What works, no hype.

Writing specs for AI coding agents

Vague specs produce vague code. When working with AI coding agents, the quality of your specifications directly determines the quality of the output. This post covers how to write specs that leave no room for interpretation.

If you’re new to spec-driven development, start with the introduction for context on what it is and which tools exist.

The problem with prose#

Vibe coding works like this: you describe what you want in conversational prose, the AI generates code, you iterate until it looks right. The problem is that natural language is ambiguous. “The system should handle invalid login attempts appropriately” means different things to different people. The AI will interpret it however its training suggests, which may not match your intent.

Spec-driven development solves the drift problem by making specifications the primary artifact.

But it raises a new question: how do you structure specs for both clarity and effectiveness?

By borrowing patterns from Behavior-Driven Development (BDD) and the RFC 2119 requirement keywords, you can write specs that are both human-readable and machine-parseable.

A structure borrowed from BDD#

Anatomy of a spec: feature, background, scenarios with GIVEN-WHEN-THEN-AND steps and RFC 2119 keywords

BDD is a software development practice that emerged from Test-Driven Development. Where TDD focuses on testing implementation, BDD focuses on specifying behavior from the user’s perspective. Teams write specifications in natural language that both stakeholders and developers can understand, then automate those specifications as tests.

Gherkin is the structured language BDD uses. It provides keywords like Feature, Scenario, Given, When, Then, and And that make specifications both readable and executable. Tools like Cucumber parse Gherkin files and run them as automated tests.

The core pattern is Given-When-Then:

Given some initial context
When an action occurs
Then expect this outcome

For spec-driven development with AI agents, we adapt this to Markdown using a Feature heading, a Background, and Scenarios written as GIVEN-WHEN-THEN-AND steps. This preserves the clarity of Gherkin while staying in a format that works everywhere.

The anatomy of a spec file#

Create one spec file per feature. Each spec file contains:

  1. Feature: A heading and a short description of what the feature does and why it matters
  2. Background: The rules and constraints that apply to every scenario
  3. Scenarios: Specific behaviors, each written as GIVEN-WHEN-THEN steps

Here’s the structure:

# Feature: Feature Name

One paragraph describing what this feature does and why it matters.

## Background

* A rule or constraint that holds for all scenarios below

## Scenarios

### Scenario: Specific behavior

* *GIVEN* a precondition
* *WHEN* a specific action occurs
* *THEN* the system SHALL do something specific
* *AND* the system SHALL also do this other thing

Every scenario needs at least one GIVEN, one WHEN, and one THEN. AND extends the step before it. For example, you can keep specs in specs/<domain>/<feature>/spec.md, so a domain groups related features.

RFC 2119 keywords#

RFC 2119 defines requirement keywords used in Internet standards. These keywords eliminate ambiguity about whether something is mandatory, recommended, or optional:

KeywordAlternativeMeaning
SHALLMUSTAbsolute requirement
SHALL NOTMUST NOTAbsolute prohibition
SHOULDRECOMMENDEDStrong recommendation, but valid exceptions may exist
SHOULD NOTNOT RECOMMENDEDStrong discouragement, but valid exceptions may exist
MAYOPTIONALTruly optional

I found that using the full set matters. Many SDD frameworks default to SHALL only. This works for core requirements, but real systems have nuance:

  • Performance optimizations that are nice to have but not critical
  • Security hardening that depends on deployment context
  • Convenience features that some users want and others don’t

Using only SHALL forces you to either make everything mandatory or omit optional behaviors entirely. The full RFC 2119 vocabulary lets you express degrees of importance.

Real examples#

Example: Calculator expression evaluation#

This example is adapted from crabculator, a terminal-based calculator.

# Feature: Expression Evaluation

The system SHALL evaluate mathematical expressions and return results.

## Background

* Expressions follow standard operator precedence

## Scenarios

### Scenario: Evaluate arithmetic expression

* *GIVEN* a valid arithmetic expression (e.g., `5 + 3 * 2`)
* *WHEN* the expression is evaluated
* *THEN* the system SHALL return the computed numeric result (e.g., `11`)

### Scenario: Evaluate expression with parentheses

* *GIVEN* an expression with parentheses (e.g., `(5 + 3) * 2`)
* *WHEN* the expression is evaluated
* *THEN* the system SHALL respect operator precedence and grouping (e.g., `16`)

### Scenario: Evaluate invalid expression

* *GIVEN* an invalid expression (e.g., `5 + + 3`, `5 / 0`)
* *WHEN* the expression is evaluated
* *THEN* the system SHALL return an error with a descriptive message
* *AND* the system SHOULD indicate the position of the error in the expression
* *AND* the system MAY suggest corrections for common mistakes

This scenario mixes all three requirement levels:

  • Returning results and errors is mandatory (SHALL)
  • Indicating error position is recommended (SHOULD)
  • Suggesting corrections is optional (MAY)

Writing effective scenarios#

Be specific about triggers#

Bad:

* *WHEN* there's an error
* *THEN* the system SHALL handle it appropriately

Good:

* *GIVEN* a query with invalid syntax
* *WHEN* the database returns error code 42000 (syntax error)
* *THEN* the system SHALL wrap it in a QuerySyntaxError
* *AND* the system SHALL include the original SQL in the error message
* *AND* the system SHALL NOT include credentials in error output

Cover edge cases explicitly#

If you don’t specify behavior for edge cases, the agent will guess:

### Scenario: Empty result set

* *GIVEN* an open connection
* *WHEN* a SELECT query returns zero rows
* *THEN* the system SHALL return an empty RecordBatch
* *AND* the system SHALL include the schema in the empty batch
* *AND* the system SHALL NOT return null or throw an exception

### Scenario: Null values in results

* *GIVEN* an open connection
* *WHEN* result data contains NULL values
* *THEN* the system SHALL preserve nulls in the Arrow representation
* *AND* the system SHALL NOT substitute default values for nulls

State what should NOT happen#

Prohibitions are as important as requirements:

### Scenario: Error logging

* *GIVEN* a connection that fails
* *WHEN* the driver logs the error
* *THEN* the system SHALL log the error type and message
* *AND* the system SHALL NOT log passwords
* *AND* the system SHALL NOT log full connection strings
* *AND* the system SHALL NOT expose stack traces in production mode

Putting it together#

Effective specs for AI coding agents:

  1. Use structured format: Feature → Background → Scenarios → GIVEN-WHEN-THEN-AND
  2. Apply RFC 2119 keywords: SHALL, SHOULD, MAY (and their negatives)
  3. Be specific: Name error codes, specify thresholds, define exact behaviors
  4. Cover edge cases: Empty results, null values, timeouts, failures
  5. State prohibitions: What the system SHALL NOT do is as important as what it SHALL do
  6. Keep specs alive: Specs should evolve with the code, not be discarded after implementation

The investment in writing precise specs pays off in reduced back-and-forth, fewer misinterpretations, and code that actually matches your intent.