This integration allows you to very simply add observability and monitoring to your LLM application based on Vercel’s AI SDK. The instrumentation is available for the 4 main methods of the Vercel AI SDK: streamText, generateText, streamObject and generateObject.
The Vercel AI SDK integration already support LLM tracing. You should not use it in conjunction with other LLM provider integrations such as OpenAI.
Copy
import { LiteralClient } from '@literalai/client';import { openai } from '@ai-sdk/openai';// Here we use the wrapped `streamText` from the Literal AI client. This will allow us to automatically capture// both the response from the LLM API and the other informations present in the response (latency, token counts, etc...).const streamText = literalAiClient.instrumentation.vercel.streamText;export async function POST(req: Request) { const { messages } = await req.json(); // The wrapped version of `streamText` accepts the same parameters as the original function. const result = streamText({ model: openai('gpt-4o'), messages, }); return result.toDataStreamResponse();}
In most cases, you will want to keep track of the different generations from your application by grouping them into Threads or Runs. This is especially useful when you want to understand the context in which a generation was made, or when you want to compare different generations.
Using our Vercel AI SDK integration, you can pass metadata, tags and a step ID at the generation level.
These values will be automatically added to the generation when it is logged on Literal AI.