Analytics and Runtime

The Runtime view provides real-time monitoring and insights into how your AI assistant is performing in production. Access analytics and conversation data to understand user interactions and optimize your assistant's performance.

Overview

Access the Runtime view by opening your project and selecting the Runtime tab. This section contains two main views:

1. Analytics - Performance metrics and usage statistics

2. Conversations - Real-time conversation monitoring and history

Analytics View

Monitor your AI assistant's performance with comprehensive analytics and metrics.

What You Can See

Overall Statistics

View key performance indicators at a glance:

• Total Conversations: Number of conversations initiated

• Total Messages: Number of messages exchanged

• Unique Users: Number of unique users who interacted with your assistant

• Tool Success Rate: Percentage of successful tool executions

Activity Over Time

Track conversation trends with time-series charts showing:

• Conversations started

• Messages sent

• Active users

• Grouped by hour, day, week, or month

Tool Executions

Monitor AI tool usage and performance:

• Total tool executions per period

• Success vs. failure rates

• Tool-specific usage breakdown

Channel Usage

See which platforms your users prefer:

• Web, Slack, Discord, Telegram, WhatsApp, API

• Message distribution across channels

• Channel-specific engagement patterns

What You Should Do

1. Choose Time Grouping

   • Select from Hour, Day, Week, or Month in the dropdown

   • Use hourly for recent detailed analysis

   • Use monthly for long-term trends

2. Refresh Data

   • Click the refresh button to load the latest analytics

   • Data updates in real-time as conversations happen

3. Analyze Trends

   • Look for usage patterns (peak hours, popular days)

   • Identify which channels are most popular

   • Monitor tool success rates for issues

4. Review Period Details

   • Scroll down to see detailed breakdowns by period

   • Each period shows:

     • Conversation count and message volume

     • User engagement metrics

     • Tool execution statistics

     • Channel and tool usage breakdowns

Use Cases

Performance Monitoring

• Track if your assistant is meeting user demand

• Identify peak usage times for capacity planning

• Monitor tool failure rates to catch issues early

User Insights

• Understand which channels users prefer

• See how many unique users are engaging

• Track average conversation duration and message count

Optimization

• Identify underperforming tools

• Spot trends in user engagement

• Make data-driven improvements to your assistant

Conversations View

View and monitor individual conversations between users and your AI assistant in real-time.

What You Can See

Conversations List (Left Panel)

Browse all conversations with:

• User name/contact information

• Assistant name

• Channel type (Web, Slack, etc.)

• Status (Active, Closed, Archived)

• Message count

• Last activity timestamp

Conversation Details (Right Panel)

When you select a conversation, view:

• Full message history with timestamps

• User messages (blue bubbles on right)

• Assistant responses (gray bubbles on left)

• Tool execution indicators

• Conversation metadata (ID, created date, status)

Message Timeline

• Messages grouped by date (Today, Yesterday, specific dates)

• Chronological order with precise timestamps

• Tool executions shown inline with messages

• Support for RTL languages (automatic detection)

What You Should Do

1. Search and Filter Conversations

   • Use the search bar to find conversations by user name, phone, email, or conversation ID

   • Filter by status: Active, Closed, or Archived

   • Filter by channel: Web, Slack, Discord, Telegram, WhatsApp, API

2. Monitor Active Conversations

   • Filter to show only "Active" conversations

   • Check recent messages to see what users are asking

   • Identify conversations that may need attention

3. Review Conversation History

   • Click any conversation to view full message history

   • Scroll up to load older messages

   • Review tool executions to understand assistant behavior

4. Analyze User Interactions

   • See how users phrase their questions

   • Review assistant responses for quality

   • Identify common conversation patterns

5. Load More Data

   • Click "Load More Conversations" to see additional conversations

   • Click "Load older messages" within a conversation to view history

   • Data loads incrementally for better performance

Use Cases

Quality Assurance

• Review assistant responses for accuracy

• Identify misunderstood user queries

• Find edge cases that need better handling

Customer Support

• Monitor active conversations

• See what issues users are experiencing

• Track conversation status and resolution

Training and Improvement

• Study successful conversations

• Identify where the assistant struggles

• Find opportunities to add new capabilities

Debugging

• View tool execution results

• Track conversation flow

• Identify technical issues

Understanding Conversation Status

• Active: Conversation is ongoing, user may send more messages

• Closed: Conversation has ended normally

• Archived: Conversation has been archived for record-keeping

Understanding Channel Types

Each conversation shows its channel with an icon:

• 🌐 Web: Browser-based chat

• 💬 Slack: Slack workspace integration

• 🎮 Discord: Discord server integration

• ✈️ Telegram: Telegram bot

• 📱 WhatsApp: WhatsApp messaging

• 🔌 API: Direct API integration

Best Practices

Analytics

1. Regular Monitoring: Check analytics daily or weekly to spot trends

2. Set Baselines: Establish normal performance metrics to identify anomalies

3. Track Changes: Monitor impact after updates to your assistant

4. Tool Success: Keep tool success rate above 95% for good UX

5. Time Grouping: Use appropriate time grouping for your use case

Conversations

1. Review Sample Conversations: Regularly read actual conversations to understand user needs

2. Filter Actively: Use filters to focus on relevant conversations

3. Check Failed Tools: Look for tool execution failures and fix root causes

4. Monitor Response Quality: Ensure assistant responses are helpful and accurate

5. Privacy: Be mindful that conversations may contain sensitive user data

Tips for Success

Performance Optimization

• If you see declining engagement, review recent conversations to understand why

• High tool failure rates indicate configuration or integration issues

• Low message counts per conversation may mean users aren't getting value

User Experience

• Monitor average conversation duration - too long may indicate confusion

• Check if users are abandoning conversations (high count of very short conversations)

• See which channels perform best and optimize accordingly

Continuous Improvement

• Use conversation data to identify new features users request

• Find common questions to add to your assistant's knowledge base

• Track before/after metrics when making changes

Troubleshooting

Analytics Not Loading

1. Click the refresh button

2. Check your internet connection

3. Verify you have permission to view this project

4. Try changing the time grouping

Conversations Not Showing

1. Check your search and filter settings

2. Click "Load More Conversations" if you expect more data

3. Refresh the page

4. Verify conversations exist in your selected filters

No Data Available

This means your assistant hasn't had any conversations yet. Once users start interacting, data will appear here.