Nov 1, 2023

Langfuse Update — October 2023

Improved dashboard, OpenAI integration, RAG evals, doubled API performance, simplified Docker deployments.

Marc Klingen

Hi everyone 👋, here's a quick overview of all the most notable new features we shipped in October:

• Improved dashboard

• OpenAI SDK integration (Python)

• Cookbook for RAG evals based on Ragas

• 2x API performance

• Simplified Docker deployments

• SDK improvements

  • SDKs expose trace URL
  • Python SDK now supports Pydantic v1 and v2

• UX improvements

  • Quickly page through traces
  • SSO via Google and GitHub
  • Delete projects

• Docs improvements: Quickly open all Python docs as Jupyter notebooks

... and many small improvements and bug fixes.

The details 👇

📈 Enhancements to dashboard

Now includes

• Costs broken down by model
• Latency percentiles by model (50th, 90th, 95th, 99th)
• Usage/cost by end-user
• Usage by trace type to break it down by application use case
• New date-range selection

🤖 OpenAI SDK integration

Drop-in replacement for OpenAI's Python SDK to get full visibility by changing only one line of code. Automatically captures: prompts, completions, usage, latencies and API errors.

- import openai
+ from langfuse.openai import openai

# Use OpenAI Python SDK as usual
openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[
      {"role": "system", "content": "You are a very accurate calculator. You output only the result of the calculation."},
      {"role": "user", "content": "1 + 1 = "}],
)

This integration fully supports grouping multiple requests into one trace if they belong to a single invocation. Also you can add scores to evaluate the outputs.

See integration docs for full details.

Thanks to @Dev-Khant (opens in a new tab) for contributing to this.

📚 Cookbook for RAG evals

Together with the Ragas (opens in a new tab) team, we've added a cookbook on how to evaluate production RAG (Retrieval augmented generation) pipelines in Langfuse. They specifically evaluate the retrieved context and the generated answer.

⚡️ Doubled API performance

API authentication previously added a significant overhead to API requests. We've now optimized this and doubled the API performance.

🐳 Simplified Docker deployments

We've integrated the application of migrations to the startup process of the Docker image. This makes it much easier to deploy and continuously update Langfuse via Docker.

docker pull ghcr.io/langfuse/langfuse:latest

Check self-hosting docs for full details on how to deploy Langfuse in a production environment.

Optionally, you can also one-click deploy Langfuse to Railway:

(opens in a new tab)

🛠 SDK improvements

Expose Langfuse trace url

In some cases, it is useful to get the trace URL directly from the SDK to add it to your logs or print it in a development environment. This is now possible in all SDKs.

# trace object
trace.get_trace_url()

# Langchain callback handler
handler.get_trace_url()

Python SDK now supports Pydantic v1 and v2

The Python SDK uses Pydantic to validate the data you send to Langfuse at run time before it is asynchronously sent to the Langfuse API. Previously, the Python SDK used Pydantic v1 leading to incompatibilities with projects using Pydantic v2.

From 1.1.3 onwards, the Python SDK supports both Pydantic v1 and v2.

🎨 UX improvements

Quickly page through traces

When you're browsing traces, you can now quickly page through them.

Use the j and k keys to navigate even faster.

Sign in via Google and GitHub

You can now authenticate using Google and GitHub. Also supported for self-hosted deployments. Check docs for details.

Delete projects

Long overdue, you can now get rid of old projects. Thanks to @DIWAKARKASHYAP (opens in a new tab) for contributing to this.

📖 Docs

"Docs is product"

All Python docs are now Jupyter notebooks so you can quickly run them. They are in the cookbook folder (opens in a new tab) of the Langfuse docs repo.

We've added a banner to the docs to make it easy to open them on GitHub and Google Colab.

Example pointing to new OpenAI integration

🚢 What's Next?

A lot is to come in November. For starters, we'll invest into the core user experience, especially to make working with complex traces easier.

If you have any feedback or feature requests on how to make Langfuse better for your user case, join us on Discord or open an issue on GitHub.

Subscribe to get monthly updates via email:

Follow along on Twitter (@Langfuse (opens in a new tab), @marcklingen (opens in a new tab))