Monitor & Performance

Track performance, costs, and user engagement through ClickAI's built-in analytics dashboard. ClickAI Monitor Dashboard

Table of Contents

· [Overview](#overview)

· [Dashboard](#dashboard)

· [Logs](#logs)

· [Annotation System](#annotation-system)

· [Integrations](#integrations)

Overview

ClickAI provides a comprehensive monitoring toolkit to help you:

· 📈 Track performance of applications in real-time

· 💰 Control costs of token and model usage

· 🐛 Debug conversation issues

· ⭐ Collect feedback from users

· ✏️ Improve quality of AI responses through annotations

Dashboard

The Dashboard provides an overview of application performance through key metrics:

Key Metrics

Metric	Description	Purpose
Total Messages	Total number of messages	App usage level
Active Users	Number of active users	User base size
Avg Response Time	Average response time	System performance
Token Usage	Tokens consumed	Operating costs
Token Cost	Token cost in currency	Budget & ROI
User Satisfaction	Satisfaction rate (👍/👎)	Response quality

Time Filters

View data across different time periods:

· Last 24 hours — Real-time monitoring

· Last 7 days — Weekly trends

· Last 30 days — Monthly analysis

· Custom range — Custom time period

Visual Charts

· Line Chart: Message and user trends over time

· Bar Chart: Daily token usage

· Pie Chart: Satisfaction rating distribution (Positive/Neutral/Negative)

💡 TIP: Monitor **Token Cost** regularly to optimize spending. If costs spike suddenly, check for looping workflows or overly long prompts.

Logs

Logs allow you to view detailed conversations, debug issues, and collect feedback.

What Gets Logged

Log Type	Details
Conversation Timeline	Chronological list of user interactions
Message Details	Full conversation context with AI responses
Performance Data	Response times and token usage per interaction
User Feedback	Ratings and comments from users and team members

Using the Logs Console

Access: Open your app → "Logs" tab

In the Logs Console, you can:

1. View timeline — Browse the list of conversations

2. Message details — Click a conversation to see full content

3. Performance data — Response time, tokens used

4. Feedback — View 👍/👎 ratings and comments

Debugging with Logs

When your AI app responds incorrectly:

5. Find the conversation with issues in Logs

6. View details — Check prompt, context, and response

7. Analyze — Identify root cause (unclear prompt, missing context, ...)

8. Improve — Adjust prompt or add knowledge

Feedback Collection

ClickAI supports collecting 2 types of feedback:

· End-user feedback: Users rate 👍/👎 on the Web App interface

· Team feedback: Team members rate and annotate directly in Logs

Log Retention

Plan	Retention Period
Sandbox	30 days
Professional & Team	Unlimited (during active subscription)
Self-hosted	Unlimited (default), configurable

Configure log retention (Self-hosted):

Environment Variable	Description
WORKFLOW_LOG_CLEANUP_ENABLED	Enable/disable auto log cleanup
WORKFLOW_LOG_RETENTION_DAYS	Number of days to retain logs
WORKFLOW_LOG_CLEANUP_BATCH_SIZE	Number of logs deleted per batch

Privacy Considerations

🛑 CAUTION: Logs contain user conversation content. Ensure compliance with data security regulations: restrict Logs access to necessary personnel only, do not share log data outside the organization, comply with PDPA/GDPR if serving international users.

Annotation System

Build a curated library of high-quality responses to improve consistency and bypass AI generation.

When to Use Annotations

· 📌 Frequently asked questions needing precise standard answers

· 🔒 Sensitive information requiring strict control

· ⚡ Want faster responses (no LLM call needed)

· 🎯 Ensure consistency for critical answers

How Annotations Work

9. User asks a question

10. System searches existing annotations for semantic matches

11. If a match above the similarity threshold is found, returns the curated response

12. If no match, proceeds with normal AI generation

13. Track which annotations get used and how often

Setting Up Annotations

14. Go to your app → Logs & Annotations → "Annotations" tab

15. Enable Annotation Reply in settings

16. Select Embedding Model for semantic matching

17. Configure Similarity Threshold (recommended: 0.7 - 0.9)

Creating Annotations

Method 1: From existing conversations

18. In Debug & Preview or Logs, find a good AI response

19. Click the "Add Annotation" icon on the response

20. Edit the answer if needed → Save

Method 2: Manual creation

21. Go to the Annotations tab

22. Click "+ Add"

23. Enter sample question and standard answer

24. Save the annotation

Method 3: Bulk import

25. In the Annotations tab, click "..." → "Bulk Import"

26. Upload a CSV file with format: question, answer

27. Review and confirm

Managing Annotation Quality

Action	Purpose
Review hit history	See which annotations are being used
Edit annotations	Update answers when information changes
Delete unused	Remove outdated annotations
Bulk export	Backup your annotation library

Integrations

ClickAI supports integration with external observability platforms:

Platform	Description
LangSmith	Tracing and debugging LLM applications
LangFuse	Open-source LLM observability
Alibaba Cloud	Alibaba Cloud monitoring integration

📝 NOTE: Third-party integrations allow you to track LLM calls, latency, and token usage in greater detail than the default Dashboard.

📖 Previous: [Publish](./02-publish-en.md) · Next: [Knowledge]