> For the complete documentation index, see [llms.txt](https://docs.clickai.vn/clickai-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.clickai.vn/clickai-docs/clickai-docs-en/developer/monitor-and-performance.md). # Monitor & Performance
## Table of Contents · \[Overview]\(#overview) · \[Dashboard]\(#dashboard) · \[Logs]\(#logs) · \[Annotation System]\(#annotation-system) · \[Integrations]\(#integrations) ## Overview ClickAI provides a comprehensive monitoring toolkit to help you: · 📈 Track performance of applications in real-time · 💰 Control costs of token and model usage · 🐛 Debug conversation issues · ⭐ Collect feedback from users · ✏️ Improve quality of AI responses through annotations ## Dashboard The Dashboard provides an overview of application performance through key metrics: ### Key Metrics
MetricDescriptionPurpose
Total MessagesTotal number of messagesApp usage level
Active UsersNumber of active usersUser base size
Avg Response TimeAverage response timeSystem performance
Token UsageTokens consumedOperating costs
Token CostToken cost in currencyBudget & ROI
User SatisfactionSatisfaction rate (👍/👎)Response quality
### Time Filters View data across different time periods: · Last 24 hours — Real-time monitoring · Last 7 days — Weekly trends · Last 30 days — Monthly analysis · Custom range — Custom time period ### Visual Charts · Line Chart: Message and user trends over time · Bar Chart: Daily token usage · Pie Chart: Satisfaction rating distribution (Positive/Neutral/Negative) 💡 TIP: Monitor \*\*Token Cost\*\* regularly to optimize spending. If costs spike suddenly, check for looping workflows or overly long prompts. ## Logs Logs allow you to view detailed conversations, debug issues, and collect feedback. ### What Gets Logged
Log TypeDetails
Conversation TimelineChronological list of user interactions
Message DetailsFull conversation context with AI responses
Performance DataResponse times and token usage per interaction
User FeedbackRatings and comments from users and team members
### Using the Logs Console Access: Open your app → "Logs" tab In the Logs Console, you can: 1\. View timeline — Browse the list of conversations 2\. Message details — Click a conversation to see full content 3\. Performance data — Response time, tokens used 4\. Feedback — View 👍/👎 ratings and comments ### Debugging with Logs When your AI app responds incorrectly: 5\. Find the conversation with issues in Logs 6\. View details — Check prompt, context, and response 7\. Analyze — Identify root cause (unclear prompt, missing context, ...) 8\. Improve — Adjust prompt or add knowledge ### Feedback Collection ClickAI supports collecting 2 types of feedback: · End-user feedback: Users rate 👍/👎 on the Web App interface · Team feedback: Team members rate and annotate directly in Logs ### Log Retention
PlanRetention Period
Sandbox30 days
Professional & TeamUnlimited (during active subscription)
Self-hostedUnlimited (default), configurable
Configure log retention (Self-hosted):
Environment VariableDescription
WORKFLOW_LOG_CLEANUP_ENABLEDEnable/disable auto log cleanup
WORKFLOW_LOG_RETENTION_DAYSNumber of days to retain logs
WORKFLOW_LOG_CLEANUP_BATCH_SIZENumber of logs deleted per batch
### Privacy Considerations 🛑 CAUTION: Logs contain user conversation content. Ensure compliance with data security regulations: restrict Logs access to necessary personnel only, do not share log data outside the organization, comply with PDPA/GDPR if serving international users. ## Annotation System Build a curated library of high-quality responses to improve consistency and bypass AI generation. ### When to Use Annotations · 📌 Frequently asked questions needing precise standard answers · 🔒 Sensitive information requiring strict control · ⚡ Want faster responses (no LLM call needed) · 🎯 Ensure consistency for critical answers ### How Annotations Work 9\. User asks a question 10\. System searches existing annotations for semantic matches 11\. If a match above the similarity threshold is found, returns the curated response 12\. If no match, proceeds with normal AI generation 13\. Track which annotations get used and how often ### Setting Up Annotations 14\. Go to your app → Logs & Annotations → "Annotations" tab 15\. Enable Annotation Reply in settings 16\. Select Embedding Model for semantic matching 17\. Configure Similarity Threshold (recommended: 0.7 - 0.9) ### Creating Annotations Method 1: From existing conversations 18\. In Debug & Preview or Logs, find a good AI response 19\. Click the "Add Annotation" icon on the response 20\. Edit the answer if needed → Save Method 2: Manual creation 21\. Go to the Annotations tab 22\. Click "+ Add" 23\. Enter sample question and standard answer 24\. Save the annotation Method 3: Bulk import 25\. In the Annotations tab, click "..." → "Bulk Import" 26\. Upload a CSV file with format: question, answer 27\. Review and confirm ### Managing Annotation Quality
ActionPurpose
Review hit historySee which annotations are being used
Edit annotationsUpdate answers when information changes
Delete unusedRemove outdated annotations
Bulk exportBackup your annotation library
## Integrations ClickAI supports integration with external observability platforms:
PlatformDescription
LangSmithTracing and debugging LLM applications
LangFuseOpen-source LLM observability
Alibaba CloudAlibaba Cloud monitoring integration
📝 NOTE: Third-party integrations allow you to track LLM calls, latency, and token usage in greater detail than the default Dashboard. *📖 Previous: \[Publish]\(./02-publish-en.md) · Next: \[Knowledge]* --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.clickai.vn/clickai-docs/clickai-docs-en/developer/monitor-and-performance.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.