Caching

Cache Invalidation Hierarchy

Change something early in a request and everything after it must be cached again, like editing page one of a book.

Definition

Prompt caches match on prefix in the order tools, system, messages, so a change invalidates its own level and everything after it. Changing tool definitions invalidates the whole cache; changing a late message only affects what follows it.

Example

Adding one tool definition invalidates the cached tools, system prompt, and messages; adding a new user message invalidates nothing before it.