Prompts

​

What is a Prompt Template?

A Prompt or Prompt Template in Literal AI consists of:

• the prompt i.e. the messages, and its variables
• the LLM provider i.e. OpenAI, Anthropic, etc.
• the LLM settings i.e. temperature, top P, etc.

Prompts in Literal AI allow you to:

• Enable product teams and domain experts to draft and iterate on prompts.
• Deploy new prompt versions without redeploying your code.
• Track which prompt version was used for a specific generation. Improve prompts by debugging logged LLM generations with context.
• Compare prompt versions, LLM models, settings and providers to determine which one performs better.
• Collaborate effectively with your team.

​

Create a Prompt

Go to the Prompt Playground to create your first prompt template. On the left panel, define the template messages. In each template message, you can define variables following the Mustache syntax.

You can try out the prompt template by setting values for the variables, picking an LLM provider/model and clicking on Submit.

The Prompt Playground supports multiple LLM providers/models and you can switch between them to see how the prompt behaves.

Once you are happy with the prompt, you click on Save.

​

Create a new version

Whether to fix a bug or to add a new feature, you can create a new version of a prompt template.

To do so, go to the Prompts page, select the prompt template you want to iterate on. Then select the version of the prompt you want to create a new version from.

Edit the template messages and click on Save when you are done. A diff of the changes will be displayed. Optionally provide a changelog to keep track of the changes.

​

Programmatically

If you prefer to keep your Prompt Template in your code, you can still version it on Literal AI. Note that A/B testing is not available if you manage prompt templates in your code.

Literal AI will check if the prompt changed compared to the last version and create a new version if needed.

​

Pull a Prompt Template from Literal AI

If your prompt templates live on Literal AI, you will have to pull them in your app before using them.

​

Format a Prompt Template

Once you have your prompt instance, you can format it with the relevant variables.

​

Format to OpenAI format

​

Convert to LangChain Chat Prompt

Coupled with integrations like OpenAI or LangChain, the Literal AI SDK will not only log the generations but also track which prompt versions were used to generate them.

This is especially useful to track the performance of your prompt versions and debug in context.

​

A/B test a Prompt

Progressively roll out new prompt versions or LLMs in production using A/B testing:

• Increases confidence in deployments by allowing gradual rollout and comparison of different versions.
• Empowers product teams to implement and test prompt improvements independently, reducing reliance on engineering teams.
• Enables data-driven decision making by comparing performance metrics between versions.
• Facilitates rapid iteration and optimization of prompts in a production environment.
• Minimizes risk by allowing easy rollback if a new version underperforms.

​

From Literal AI

When pulling a prompt template without specifying a version, Literal AI will use the A/B testing rollout probabilities to select the version to use.

By default, version v0 has a 100 % rollout. You can change the rollout splits from the Prompt details page page.

​

From Code

You can also update the A/B testing rollout probabilities from code:

​

Iterate on Prompts

You can test a specific version of a prompt against a dataset and a set of scorers to measure its performance and avoid regressions. You can find more information in the experiments guide.