System Prompt: The Instructions an App Gives an AI Model

Instructions that guide the model’s responses

A system prompt is a set of instructions and background information that an app gives an AI model to guide how it responds. It can describe the model’s role and task, give rules it should follow and the tone to use, show what a good answer looks like, and supply background such as company policies.

That’s how one model can work as quite different assistants. A shop’s returns bot, an English tutor and a tool that summarizes legal documents can all run on the same model, each acting differently because its app gives it different instructions. Put simply, the system prompt carries the app’s guidance and your message carries your request, though both can shape the answer.

The app sends it separately from the messages you type, and you usually don’t see it. Two providers’ documentation describes these instructions as high priority or ranks them ahead of a user’s messages, though priority isn’t a guarantee.

Services name these instructions differently (system prompt, system instructions, system message, developer messages), and one provider ranks its own rules above the app’s, as the section on files and ranks below explains. This piece uses “system prompt” as an umbrella term.

Following a chat request by request

Here’s a short exchange with a shop’s support chatbot, without any tools, as an example.

You ask: “Can I return a jacket I bought last month?” The app sends instructions such as “You are our shop’s returns assistant. Returns are accepted within 30 days of delivery. Keep answers brief and friendly.” Separately, it sends your question, and the model uses both to write its reply.

Then you ask “What if it’s been 35 days?” and attach your receipt. The app sends another request: in this example the same system prompt, then your first question, the first reply and your new message, with the receipt inside that message. The model uses all of it to reply.

Why the same instructions twice? In this example, each request includes the instructions and the conversation so far, and each call produces one reply. On the developer interfaces of two major providers, each call stands on its own, so the conversation has to be supplied every time: an app can send it along itself, as this one does, or let the provider’s service store it and supply it. Everything supplied for a call, plus what the model writes, has to fit in the model’s context window, its working memory, which has a limit, and the system prompt uses some of that space.

The app could also send different instructions later, and on some models it can add new system instructions partway through a conversation. Long conversations may be shortened or summarized to fit, and with tools one answer can take several calls to the model.

If a picture helps, imagine a stand-in at a front desk who remembers nothing between calls. Each call comes with a fresh folder: a front pocket labeled “Instructions for the assistant” holds the instructions and some background notes, and behind it sits the call so far, including any papers the caller has handed over. The caller normally doesn’t see the front pocket.

Files, ranks and leaks

Uploading a file doesn’t make it a system prompt. If the app sends its contents inside your message, as with the receipt, they’re part of the conversation. An app can also put text from a file into its system prompt as background information. Tools can bring material in too, as tool results.

With one provider, the app’s instructions outrank a user’s conflicting instructions, and the provider can set rules of its own above them that, by the provider’s own account, the app’s builder may not know. Priority doesn’t promise obedience, though: another provider’s documentation for one of its models warns that a chatbot can stop following its system prompt when users argue or keep asking.

Being out of view doesn’t make it safe, either. One provider’s documentation warns that no way of preventing prompt leaks is foolproof, and advises leaving out details the model doesn’t need.

The folder misleads in a few places. The pocket stands for a set-aside field or message in the request, and typing “System prompt” at the top of your own message doesn’t change its role or move it there. The pocket isn’t only for companies: someone building an app for themselves fills it too. A pocket doesn’t guarantee its contents get followed. And some services store the conversation for the app instead of handing over a fresh folder each time.

A chatbot stuck on old terms

Take a made-up retailer (the company and the numbers are invented for this example). It cuts its return window from 60 days to 30 and updates the policy page on its website, yet months later its support chatbot still promises 60 days.

The contractor who set the bot up had written the old return window into its system prompt. Updating the website didn’t change what the model received, and nobody owned the system prompt.

Who should own it

A company-controlled system prompt can carry policy information and instructions. Release a changed version and you change what the model receives wherever that version is used, which makes approving it a business decision.

Our recommendation: give every system prompt your company controls a business owner (when it carries policy, the person who owns that policy), with the technical team testing changes before release, as one provider’s documentation also recommends. Keep a short record of which policies live in a system prompt and which in documents the tool reads, so a policy change comes with a list of places to update. For facts that change, such as a return window, a maintained source the app reads is a better home than a copy in the system prompt, once you’ve checked that the app really reads the updated version. And for rules that must hold, add a control outside the model, such as a returns system that won’t approve a return after 30 days, for the same reason the stop belongs outside the agent.

A bought tool’s system prompt may be the vendor’s. One provider, for instance, publishes the system prompt behind its own chat apps and updates it from time to time.

In short: use the system prompt to guide behavior, a maintained source for facts that change, and a control outside the model for rules that must hold.

Checking your own setup

1. Which of our AI tools have a system prompt we control, and how do we hear when a vendor changes theirs?
2. Who owns each one, and who approves a change before it goes live?
3. Which policies live in a system prompt, and which in documents the tool reads?
4. Is there anything in our system prompts we wouldn’t want a customer to see?
5. Which rules must hold even if the tool ignores its instructions, and what enforces them outside the tool?