ObservabilityConfig

Configure monitoring and tracing with Langfuse

Overview

ObservabilityConfig enables monitoring and tracing of your RAG pipeline using Langfuse.

Definition

from dataclasses import dataclass
from typing import Optional

@dataclass
class ObservabilityConfig:
    """Configuration for observability/monitoring."""
    enabled: bool = False
    public_key: Optional[str] = None
    secret_key: Optional[str] = None
    host: Optional[str] = None

Fields

enabled · bool · default: "False"

Whether to enable observability tracking

public_key · Optional[str] · default: "None"

Langfuse public key (defaults to LANGFUSE_PUBLIC_KEY env var)

secret_key · Optional[str] · default: "None"

Langfuse secret key (defaults to LANGFUSE_SECRET_KEY env var)

host · Optional[str] · default: "None"

Langfuse host URL (defaults to LANGFUSE_HOST env var or Langfuse cloud)

Usage

Basic Usage

from mini import AgenticRAG, ObservabilityConfig

rag = AgenticRAG(
    vector_store=vector_store,
    embedding_model=embedding_model,
    observability_config=ObservabilityConfig(
        enabled=True
    )
)

Complete Configuration

import os
from mini import ObservabilityConfig

observability_config = ObservabilityConfig(
    enabled=True,
    public_key=os.getenv("LANGFUSE_PUBLIC_KEY"),
    secret_key=os.getenv("LANGFUSE_SECRET_KEY"),
    host="https://cloud.langfuse.com"
)

rag = AgenticRAG(
    vector_store=vector_store,
    embedding_model=embedding_model,
    observability_config=observability_config
)

Environment Variables

Set up your .env file:

LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_HOST=https://cloud.langfuse.com  # Optional

Then use in code:

from dotenv import load_dotenv
load_dotenv()

# Automatically uses environment variables
observability_config = ObservabilityConfig(enabled=True)

What Gets Tracked

When enabled, Mini RAG tracks:

Query Operations

• Query text and variations (if query rewriting enabled)
• Query execution time
• Retrieved chunks and scores
• Reranking results
• Final answer generation

Document Indexing

• Document loading operations
• Chunking results
• Embedding generation
• Vector store insertions

LLM Calls

• Model used
• Tokens consumed
• Latency
• Input and output

Performance Metrics

• End-to-end latency
• Individual component timings
• Token usage and costs
• Success/failure rates

Setup Langfuse

1. Create Account

   Sign up at Langfuse Cloud (free tier available)

2. Create Project

   Create a new project in Langfuse dashboard

3. Get API Keys

   Navigate to Settings → API Keys and copy your keys

4. Set Environment Variables

   Add keys to your .env file:

   LANGFUSE_PUBLIC_KEY=pk-lf-...
   LANGFUSE_SECRET_KEY=sk-lf-...
   
5. Enable in Code

   observability_config = ObservabilityConfig(enabled=True)
   

Benefits

Debugging

See exactly what happens in each query

Performance

Track latency and identify bottlenecks

Cost Tracking

Monitor LLM token usage and costs

Quality Analysis

Analyze retrieval and answer quality

Production Example

import os
from mini import (
    AgenticRAG,
    LLMConfig,
    RetrievalConfig,
    ObservabilityConfig,
    EmbeddingModel,
    VectorStore
)
from dotenv import load_dotenv

load_dotenv()

# Initialize with observability
rag = AgenticRAG(
    vector_store=VectorStore(...),
    embedding_model=EmbeddingModel(),
    llm_config=LLMConfig(model="gpt-4o-mini"),
    retrieval_config=RetrievalConfig(
        top_k=10,
        rerank_top_k=3,
        use_query_rewriting=True,
        use_reranking=True
    ),
    observability_config=ObservabilityConfig(
        enabled=True
    )
)

# All operations are automatically tracked
response = rag.query("What is the budget?")
rag.index_document("document.pdf")

# View traces in Langfuse dashboard

Disabling Observability

To disable observability (e.g., for local development):

# Disabled by default
rag = AgenticRAG(vector_store=vector_store, embedding_model=embedding_model)

# Or explicitly disable
observability_config = ObservabilityConfig(enabled=False)

See Also

Observability Guide

Learn more about observability

LangFuse Docs

Langfuse documentation

Production Guide

Deploy to production