2024/05/14

Monitoring LLM Security in Langfuse

How to use Langfuse to trace, prevent, and evaluate security risks common to LLM-based applications.

Lydia You

Marc Klingen

Protecting against security risks and attacks is becoming increasingly important for ensuring LLM apps are production ready. Not only do LLM applications need to be secure to protect users' private and sensitive information, they also need ensure a level of quality and safety of responses to maintain product standards.

This post offers an overview of how you can use security tools in conjunction with Langfuse to monitor and protect against common security risks. The OWASP Top 10 (opens in a new tab) list is a useful resource on the topic. It provides a consensus of the most critical security risks for LLM applications.

Introduction

In the video below, we walk through an example of how to use the open-source security library LLM Guard, and how to integrate Langfuse to monitor and protect against common security risks.

How to Monitor LLM Security

1. Run-time security measures with LLM security libraries

There are several popular security libraries that can be used to mitigate security risks in LLM-based applications. These include: LLM Guard (opens in a new tab), Prompt Armor (opens in a new tab), NeMo Guardrails (opens in a new tab), Microsoft Azure AI Content Safety (opens in a new tab), Lakera (opens in a new tab). These libraries help with security measures in the following ways:

1. Catching and blocking a potentially harmful or inappropriate prompt before sending to the model
2. Redacting sensitive PII before being sending into the model and then un-redacting in the response
3. Evaluating prompts and completions on toxicity, relevance, or sensitive material at run-time and blocking the response if necessary

2. Asynchronous monitoring and evaluation of security measures with Langfuse

Use Langfuse tracing to gain visibility and confidence in each step of the security mechanism. These are common workflows:

1. Investigate security issues by manually inspect traces.
2. Monitor security scores over time in the Langfuse Dashboard.
3. Evaluate effectiveness of security measures. Integrating Langfuse scores into your team's workflow can help teams identify which security risks are most prevalent and build more robust tools around those specific issues. There are two main workflows to consider:
   • Annotations (in UI). If you establish a baseline by annotating a share of production traces, you can compare the security scores returned by the security tools with these annotations.
   • Automated evaluations. Langfuse's model-based evaluations will run asynchronously and can scan traces for things such as toxicity or sensitivity to flag potential risks and identify any gaps in your LLM security setup. Check out the docs to learn more about how to set up these evaluations.
4. Track latency to balance tradeoffs. Some LLM security checks need to be awaited before the model can be called, others block the response to the user. Thus they quickly are an essential driver of overall latency of an LLM application. Langfuse can help disect the latencies of these checks within a trace to understand whether the checks are worth the wait.

Example Workflow: Anonymizing Personally Identifiable Information (PII)

We redact and un-redact sensitive information using a security library before and after it is fed into the model. We wrap the whole process with the Langfuse observe decorator to trace and monitor the security process. In the following example below we use the open source library LLM Guard (opens in a new tab), an open-source security tool. All examples easily translate to other libraries.

Exposing Personally Identifiable Information (PII) to models can pose security and privacy risks, such as violating contractual obligations or regulatory compliance requirements, or mitigating the risks of data leakage or a data breach.

The example below shows a simple application that summarizes a given court transcript. For privacy reasons, the application wants to anonymize PII before the information is fed into the model, and then un-redact the response to produce a coherent summary.

pip install llm-guard langfuse openai

from llm_guard.input_scanners import Anonymize
from llm_guard.input_scanners.anonymize_helpers import BERT_LARGE_NER_CONF
from langfuse.openai import openai # OpenAI integration
from langfuse.decorators import observe, langfuse_context
from llm_guard.output_scanners import Deanonymize
from llm_guard.vault import Vault

vault = Vault()
 
@observe()
def anonymize(input: str):
  scanner = Anonymize(vault, preamble="Insert before prompt", allowed_names=["John Doe"], hidden_names=["Test LLC"],
                    recognizer_conf=BERT_LARGE_NER_CONF, language="en")
  sanitized_prompt, is_valid, risk_score = scanner.scan(prompt)
  return sanitized_prompt
 
@observe()
def deanonymize(sanitized_prompt: str, answer: str):
  scanner = Deanonymize(vault)
  sanitized_model_output, is_valid, risk_score = scanner.scan(sanitized_prompt, answer)
 
  return sanitized_model_output

@observe()
def summarize_transcript(prompt: str):
  sanitized_prompt = anonymize(prompt)
 
  answer = openai.chat.completions.create(
        model="gpt-3.5-turbo",
        max_tokens=100,
        messages=[
          {"role": "system", "content": "Summarize the given court transcript."},
          {"role": "user", "content": sanitized_prompt}
        ],
    ).choices[0].message.content
 
  sanitized_model_output = deanonymize(sanitized_prompt, answer)
 
  return sanitized_model_output

prompt = """
Plaintiff, Jane Doe, by and through her attorneys, files this complaint
against Defendant, Big Corporation, and alleges upon information and belief,
except for those allegations pertaining to personal knowledge, that on or about
July 15, 2023, at the Defendant's manufacturing facility located at 123 Industrial Way, Springfield, Illinois, Defendant negligently failed to maintain safe working conditions,
leading to Plaintiff suffering severe and permanent injuries. As a direct and proximate
result of Defendant's negligence, Plaintiff has endured significant physical pain, emotional distress, and financial hardship due to medical expenses and loss of income. Plaintiff seeks compensatory damages, punitive damages, and any other relief the Court deems just and proper.
"""
summarize_transcript(prompt)

Get Started

Run the end-to-end cookbook or check out our documentation.

About Langfuse

Langfuse is the open source LLM engineering platform. It is used by teams to track and analyze their LLM app in production with regards to quality, cost, and latency across product releases and use cases.

Feedback / Questions?

Join us on GitHub Discussions!