​
Monitoring Your AI Application

Effective monitoring is crucial for maintaining and optimizing your AI application. This guide will walk you through the key aspects of monitoring, including using the dashboard, understanding important metrics, and accessing logs.

Monitoring is closely related to evaluation and continuous improvement.

​
Dashboard Overview

The dashboard provides a comprehensive view of your AI application’s performance and usage.

Dashboard

Here are the main sections you’ll find:

​
Volume Metrics

Track the usage and activity of your AI application over time:

  • Number of conversation threads
  • Agent runs
  • Text generations
  • Token usage
  • User feedback submissions

​
Latency Metrics

Monitor the speed and responsiveness of your AI:

  • Time to first token: How quickly your AI starts generating a response
  • Token throughput: The rate at which tokens are generated

​
AI Performance Evaluations

Configure AI Evaluations

Evaluate the quality and effectiveness of your AI:

  • Set up AI evaluations in production to continuously monitor performance
  • For detailed information on setting up evaluations, refer to our evaluation guide

​
Cost Tracking

Keep an eye on the financial aspects of your AI application:

1

Access LLM Settings

Navigate to the “Settings” section and select the “LLM” tab.

2

Configure Model Costs

Configure the cost per token for each model you’re using to enable precise cost tracking:

  • Pattern: Specify a regular expression to match the model name (e.g., “gpt-4o*” for all GPT-4o variants)
  • Input Price: Set the cost for input tokens in USD per million tokens
  • Output Price: Set the cost for output tokens in USD per million tokens
  • Period: Optionally define start and end dates for time-specific pricing (useful for handling price changes)

You can add multiple model configurations to accurately track expenses across your LLM providers.

3

Monitor Costs in the Dashboard

Return to the main dashboard to view cost metrics over time, including total cost, cost per conversation, and cost breakdowns by model.

Cost Metrics in the Dashboard

​
Accessing Logs

Detailed logs provide valuable insights for troubleshooting and optimization:

  1. In the sidebar, navigate to the “Logs” section
  2. Use filters to narrow down logs by date, conversation ID, or AI eval results
  3. Review log entries for specific conversations or errors to identify root causes

Was this page helpful?

Annotation QueuesContinuous Improvement