LangChain/LangGraph - Literal AI Documentation

The Langchain integration enables to monitor your Langchain agents and chains with a single line of code.

You should create a new instance of the callback handler for each invocation.

You should call literalai_client.flush() at the end of your script to ensure all generations are logged.

The LangChain integration already support LLM tracing. You should not use it in conjunction with other LLM provider integrations such as OpenAI.

import os
from literalai import LiteralClient

from langchain_openai import ChatOpenAI
from langchain.schema.runnable.config import RunnableConfig
from langchain.schema import StrOutputParser
from langchain.prompts import ChatPromptTemplate

literalai_client = LiteralClient(api_key=os.getenv("LITERAL_API_KEY"))
cb = literalai_client.langchain_callback()

# optional: attach to a prompt template on Literal AI
# prompt = literalai_client.api.get_or_create_prompt(...)
# prompt_template = prompt.to_langchain_chat_prompt_template()

prompt_template = ChatPromptTemplate.from_messages(
    ['human', 'Tell me a short joke about {topic}']
)

model = ChatOpenAI(streaming=True)
runnable = prompt_template | model | StrOutputParser()

res = runnable.invoke(
    {"topic": "ice cream"},
    config=RunnableConfig(callbacks=[cb], run_name="joke")
    )

import { LiteralClient } from '@literalai/client';

import { StringOutputParser } from '@langchain/core/output_parsers';
import { ChatPromptTemplate } from '@langchain/core/prompts';
import { ChatOpenAI } from '@langchain/openai';

const literalAiClient = new LiteralClient({apiKey: process.env["LITERAL_API_KEY"]}); // This is the default and can be omitted

const cb = literalAiClient.instrumentation.langchain.literalCallback({
  // You can add a list of chain types you want to ignore on Literal AI
  chainTypesToIgnore: ["ChatPromptTemplate"],
});

const prompt = ChatPromptTemplate.fromMessages([
    ['human', 'Tell me a short joke about {topic}']
  ]);

const model = new ChatOpenAI({});
const outputParser = new StringOutputParser();

const chain = prompt.pipe(model).pipe(outputParser);

async function main() {
  const response = await chain.invoke(
    { topic: "ice cream" },
    {
      // If you give your run a LangChain run name, it will also be named like that in Literal AI
      runName: "That one Ice Cream joke",
      callbacks: [cb],
    }
  );
}
main();

Multiple langchain calls in a single thread

You can combine the Langchain callback handler with the concept of Thread to monitor multiple langchain calls in a single thread.

import os
from literalai import LiteralClient

literalai_client = LiteralClient(api_key=os.getenv("LITERAL_API_KEY"))

with literalai_client.thread(name="Langchain example") as thread:
    cb = literalai_client.langchain_callback()
    # Call your Langchain agent here

import { LiteralClient } from '@literalai/client';

const literalAiClient = new LiteralClient();

// You have three options to group multiple langchain calls in a single thread

/**
 * The first option is to add a threadId to the callback. Every time you run this code, a new
 * thread called "Jokes" will be created
*/
const cb = literalAiClient.instrumentation.langchain.literalCallback({
  threadId: "Jokes",
});

async function main() {
  const response = await chain.invoke(
    { topic: "ice cream" },
    { callbacks: [cb] }
  );
}
main();

/**
 * The second option is to use LangChain's built-in thread_id concept.
 * This will create a persistent thread, meaning every time you run this code, the new runs
 * and generations will be stacked inside the existing thread
 */
const cb = literalAiClient.instrumentation.langchain.literalCallback();

async function main() {
  const response = await chain.invoke(
    { topic: "ice cream" },
    {
      callbacks: [cb],
      configurable: { thread_id: "Jokes" },
    }
  );
}

/**
 * Alternatively, you can still use the Literal AI thread or step wrapper to group multiple langchain calls
 * Like in the first option, this will create a new thread every time you run this code
*/

async function main() {
  await literalAiClient.thread({ name: "Jokes" }).wrap(async () => {
    const response = await chain.invoke(
      { topic: "ice cream" },
      {
        callbacks: [cb],
        configurable: { thread_id: "Jokes" },
      }
    );
  });
}

Adding tags, metadata or a Step ID

If you use LangChain’s built-in tags and metadata, they will be added to the Literal AI generations. Additionally, you can specify a Step ID to ensure a generation is logged with this Step ID.

import { v4 as uuidv4 } from 'uuid';
import { ChatOpenAI } from '@langchain/openai';

const client = new LiteralClient({ apiKey, apiUrl });

const cb = client.instrumentation.langchain.literalCallback();

const model = new ChatOpenAI({});

const literalaiStepId = uuidv4();


await model.invoke('Hello, how are you?', {
  callbacks: [cb],
  metadata: {
    key: 'value',
    // use literalaiStepId in the metadata to specify a Step ID
    literalaiStepId,
  },
  tags: ['tag1', 'tag2'],
});

LangGraph

LangGraph works similarly to LangChain when it comes to using the Literal AI callback handler.

# list imports 

workflow = StateGraph(MessagesState)
# define graph ...

app = workflow.compile(checkpointer=checkpointer)

# run the app with LangChain callback handler
cb = literalai_client.langchain_callback()
final_state = app.invoke(
    {"messages": [HumanMessage(content="what is the weather in sf")]},
    config=RunnableConfig(callbacks=[cb])
)

const cb = literalAiClient.instrumentation.langchain.literalCallback();

// Define graph, nodes and edges
const graphState: StateGraphArgs<AgentState>["channels"] = {
  // ...
};

const workflow = new StateGraph<AgentState>({ channels: graphState })
  .addNode(...);

const app = workflow.compile();

// Invoke the graph with the callback handler
const finalState = await app.invoke(
  { messages: [new HumanMessage("what is the weather in sf")] },
  {
    configurable: { thread_id: "Weather Thread" },
    runName: "weather",
    callbacks: [cb],
  }
);

Python

Check out this Python LangGraph example.

Typescript

Check out this Typescript LangGraph example.

Link a Literal AI Prompt to a LangChain/LangGraph Run

prompt = literalai_client.api.get_prompt(name="RAG prompt")
langchain_prompt = prompt.to_langchain_chat_prompt_template()
# Use langchain_prompt as any other LangChain prompt

const prompt = await literalAiClient.api.getPrompt("RAG prompt");
const chatPrompt = prompt.toLangchainChatPromptTemplate();
// Use chatPrompt as any other LangChain prompt

The Literal AI SDK will not only log the generations but also track which prompt versions were used to generate them. This is especially useful to track the performance of your prompt versions and debug in context.

​Multiple langchain calls in a single thread

​Adding tags, metadata or a Step ID

​LangGraph

Python

Typescript

​Link a Literal AI Prompt to a LangChain/LangGraph Run

Multiple langchain calls in a single thread

Adding tags, metadata or a Step ID

LangGraph

Link a Literal AI Prompt to a LangChain/LangGraph Run