Rules for Rules: Writing Docs for LLMs

Documentation for developer tools is changing rapidly. As LLMs become a prominent (and likely even the primary) way that developers learn about new technologies, it becomes critical to be able to write effective documentation for LLMs to consume. If you’re building an open-source library, a framework, or an API-based service then part of your job is now to provide a way for users to get effective guidance about your product when working with AI.

One of the best/easiest ways to do this right now is by providing “rules” files. Most AI-assisted coding environments (e.g. Firebase Studio, Cursor, Cline, Windsurf, Copilot) have a way for users to supply their own custom guidance to the model. By providing concise, LLM-friendly “rules” for your library, you can make it easy for developers to copy-and-paste their way to more effective use.

Writing high-quality LLM-friendly documentation can help fight what I call “The Great Ossification” where incumbent popular technologies have a durable unfair advantage because they have more examples in the training data. You can give your users the ability to vibe with your code — even if it’s new, even if it’s different — by writing great LLM guidance.

General Process

1. Come up with a list of common ways your product might need code generated by the LLM.
2. Using your favorite prompt testing environment (I like AI Studio or Dotprompt Fiddle), try out a few basic variations of prompting for common use cases of your library and review the output.
3. Try against a few different models and try to identify some strengths and weaknesses. Try prompting for relatively complex examples to push the limits.
4. Start writing your rules. You can use your product’s documentation as a guide, but try to be extremely judicious in cutting content for length. Ideally you want to burn as few tokens as possible on your rules.
5. Try out some of your problematic examples with the rules you wrote. See if they effectively address the issue you saw. If so, try to think of a few variations to test.
6. Repeat with new examples, tweaking and iterating until you get fairly reliable output.
7. Try weaker models to see how your guidance holds up. If Gemini 2.5 Pro can code with your rules that’s nice (and the bare minimum!), but if Gemini 2.5 Flash can do it, even better. If 2.5 Flash Lite works, you’re really cooking!

Rules for Rules

To some extent you’ll need to “unlearn” some of the practices you’re used to when writing documentation for humans. These are some formatting and structural tips that will keep your rules concise, clear, and comprehensible for an LLM.

Granularity

✅ Create multiple files with different granularity.
⛔ Create a sprawling “kitchen sink” rules file.

The whole point of LLM Rules files are to provide relevant contextual guidance that helps a user in a specific situation. If your library/tool/service has a relatively small surface area, you might try to bundle everything together. But if it doesn’t, you’ll be better off creating multiple rules files for different situations.

An obvious example of this: if your product can be used in multiple programming languages, you need a minimum of one rules file for each language since a given user is unlikely to be using multiple at once.

Background

✅ Describe when/how.
⛔ Describe what/why (at length).

LLMs are trained on the entire internet and have vast amounts of approximate knowledge. An LLM is like lossy compression of all human knowledge. LLMs already “understand” the vast majority of programming concepts, popular tools/libraries, etc.

When you are writing LLM rules you aren’t telling it what something is or why it should care, you are telling it how to hold it correctly. Provide concise text about when the LLM might want to use the code you’re showing, and how it should write code in that situation.

All of the conceptual scaffolding you might need to provide for a human, especially a beginner, is unnecessary. Think of the LLM as an engineer who is familiar with the domain of your product/library/tool but either hasn’t used it for a few years or has only used other similar things and not this specific thing.

Reference Docs

✅ Document options directly in examples.
⛔ Exhaustively regurgitate reference docs as bulleted items.

LLMs are great at understanding code, including comments in code. Rather than providing extensive “formatted text” style documentation, you can supply targeted in-context scenarios that are close to the format the LLM will actually need to generate.

Positivity

✅ Provide positive examples of what to do.
⛔ Provide negative examples (most of the time).

LLMs have a real “don’t think about elephants” problem. By putting something into context, even if your text says “Don’t do X”, you are priming the LLM to “think” about X. Whenever possible, reframe examples in terms of positive best practices, not problems to avoid.

Negative examples should only be used as a last resort, largely to prompt around stubborn API usage issues with e.g. breaking changes between major versions (see below).

This is one of the hardest guidelines to follow. It’s natural for us to speak in terms of “do this, not that” (this whole post does so!).

Model Quirks

✅ Specify version information and recent syntax changes.
⛔ Assume all models work the same.

When introducing your documentation, specify the version your documentation targets (e.g. “Genkit 1.x”) to help ground the model on how the examples might be different from training data.

It’s important to test the model out without any prompting to uncover specific outdated API use. Provide “patches” for these in a very limited and targeted fashion. Dumping an entire migration guide might lead to “don’t think about elephants” issues.

Few-shot Examples

✅ Provide a few larger in-context examples.
⛔ Overdo it with multi-shot examples.

Models are getting pretty smart. It used to be that you might need to provide a bunch of different examples for the model to get the idea, but that’s less and less the case.

It’s good to provide one or two larger examples of a complete use case — a kind of “in conclusion, it will end up looking like this”. But don’t overdo it by providing many examples with minimal variation or it can lead to the model overfitting to the example set.

Structure

✅ Use “XML tags” to provide structured labels to the rules.
⛔ Think you’re actually writing XML.

A common technique in LLM prompting is to use XML opening and closing tags to label and identify content. This can help the LLM disambiguate content in various situations.

You can use this to your advantage such as wrapping your examples in <example> and </example> tags, but don’t think you’re actually writing an XML document, because you’re not. Don’t deeply nest tags, don’t put the whole rules file in a tag. Use them simply and sparingly.

The Rules for Rules Rules

To show you an example of what I mean, let me try to rewrite this post as “LLM Rules”:

# LLM Rules Generation Guidance

Your goal is to generate concise, high-quality documentation that an LLM can consume to reliably and accurately leverage a developer product.

- For small API surface areas, generate a single concise rules files. For large API surfaces, generate a non-exhaustive "primary" rules file and multiple topic-specific "secondary" rules files.
- For any product that supports multiple programming languages, generate separate rules files for each language.
- Generate concise usage scenarios and code samples with minimal narrative text.
- Document interfaces and options directly in code via commented alternatives. For example:

<example>
```ts
// brief description of when to use ExampleClass
const exampleClass = new ExampleClass({
  // concise documentation about option1
  option1: true,
  option2: {value: 'abc'}, // documentation about when to use this version of option2
  option2: 12345, // documentation about when to use the alternate version of option2
});
```
</example>

- Add best practices examples for important concepts, focusing on correct use.
- Specify the version of the library in the guidance to ground against training.
- Provide negative API examples ONLY for specific issues observed in test generation. When providing negative examples, concisely describe correct and incorrect syntax:

<example>
```ts
const exampleClass = new ExampleClass({...}); // CORRECT 2.x syntax
const exampleClass = ExampleClass.load({..}); // INCORRECT 1.x syntax
``
</example>

- Conclude a rules file with two larger examples that utilize multiple concepts together. The examples should be distinct and exercise different parts of the API.
- Use XML tags (e.g. <example>) to wrap specific sections when it might otherwise be ambiguous. XML tags should generally have their own line.

As you can see, it conveys much of the same information as the post, but is “compressed” toward guiding behavior over explaining rationale.

Do you know any “Rules for Rules” I’m missing? Have you found other effective techniques for LLM-friendly documentation? Let me know!