Skip to content

Observability Portal

Observability Portal

The Observability Portal provides comprehensive monitoring, logging, and analytics for all your LLMInspect activities. Access it at https://llminspect-observability.eunomatix.com/ using your LLMInspect credentials.

Dashboard Overview

Accessing Your Dashboard

  1. Log in to the Observability Portal using your credentials

  2. Navigate to Dashboards from the left pane

  3. Select User from the dashboard categories

  4. Choose LLMInspect to view your comprehensive dashboard

Dashboard Navigation

Dashboard Panels

Model Usage Panel

This panel displays AI model utilization ranked by usage frequency, helping you understand which models are most popular within your organization.

Features:

  • Models ranked from most to least used

  • Visual representation of usage distribution

  • Click-through capability for detailed model statistics

Use Cases:

  • Identify preferred models for budget planning

  • Understand model adoption patterns

  • Optimize model availability based on demand

User Messages and Sentiments Table

A comprehensive table showing detailed information about each user interaction with validation results.

Table Columns:

  • Date: Timestamp of the interaction

  • Prompt: User's input message (may be truncated for display)

  • IPv4: Source IP address of the request

  • User Agent: Browser/client information

  • Validations: Visual indicators for each security validation:

    • ✅ Green Tick: Validation passed
    • ❌ Red X: Validation failed (blocked)
    • ⚠️ Exclamation Mark: Validation not active/skipped

User Messages Table

Benefits:

  • Quick identification of blocked content

  • Audit trail for compliance

User Activity Statistics Panel

Displays key metrics for individual user activity and security compliance.

Metrics Shown:

  • Name: User identifier

  • Total Messages: Complete count of user interactions

  • Failed Validations: Number of security blocks/warnings

Applications:

  • Identify users who may need additional training

  • Monitor compliance across team members

  • Track individual usage patterns

Image Generation Metrics

Tracks the volume of AI-generated images across your organization.

Displays:

  • Total number of images generated

  • Usage patterns for image generation features

Average Tokens Per User

Shows token consumption patterns to help with usage optimization and cost management.

Features:

  • Per-user token averages

  • Comparison across team members

  • Historical trending data

Benefits:

  • Cost allocation and budgeting

  • Identify heavy usage patterns

  • Optimize token usage efficiency

Most Used Model Panel

Highlights the single most popular AI model in your organization.

Shows:

  • Current most popular model

  • Usage percentage or count

  • Historical comparison data

Files by User Panel

Tracks document upload and processing activities across users.

Metrics:

  • Number of files processed per user

  • File processing trends

  • User engagement with document features

Use Cases:

  • Monitor document processing adoption

  • Identify power users of file features

  • Track data input patterns

Data Volume Panel

Displays overall data throughput and processing volumes.

Includes:

  • Total data processed

  • Volume trends over time

  • Peak usage periods

Data Volume Panel

Using Dashboard Data

Performance Analysis

Model Optimization:

Use the Model Usage and Most Used Model panels to:

  • Allocate resources to popular models

  • Phase out underutilized models

  • Plan for capacity scaling

User Training:

Leverage User Activity Statistics and Messages Table to:

  • Identify users with high validation failures

  • Provide targeted security awareness training

  • Improve prompt engineering skills

Security Monitoring

Validation Tracking:

The User Messages and Sentiments Table provides:

  • Real-time security event monitoring

  • Pattern identification for attack attempts

  • Compliance audit trails

Risk Assessment:

Use Failed Validations metrics to:

  • Identify security hotspots

  • Adjust GuardRail sensitivity

  • Plan additional security measures

Resource Management

Cost Control:

Monitor resource usage through:

  • Average Tokens Per User for cost allocation

  • Data Volume for infrastructure planning

  • Image Generation for specialized resource needs

Capacity Planning:

Use dashboard data to:

  • Predict peak usage periods

  • Scale infrastructure proactively

  • Optimize resource allocation

Dashboard Navigation Tips

Filtering and Time Ranges

Most panels support:

  • Time Range Selection: View data for specific periods

Drill-Down Capabilities

  • Click on panel elements for detailed views

  • Export data for external analysis

Refresh and Real-Time Data

  • Dashboard auto-refreshes for near real-time monitoring

  • Manual refresh options for immediate updates

  • Historical data retention for trend analysis