Amazon Bedrock

Trace AWS Bedrock Converse API calls in Muster via the @observe decorator or via framework integrations.

Amazon Bedrock is a fully managed AWS service that gives you access to foundation models from Anthropic, Meta, Mistral, Cohere, and others through a single API. With Muster you can capture detailed traces and metrics for every Bedrock request — including playground, LLM-as-a-judge, and prompt experiment compatibility once your Bedrock credentials are added in project settings.

Integration options

There are several ways to capture Bedrock calls:

Through a framework already integrated with Muster: LangChain, LlamaIndex, Haystack, Vercel AI SDK.
Through a proxy — LiteLLM.
By wrapping the Bedrock SDK with the Muster @observe() decorator (example below).

Wrapping the Bedrock Converse API

%pip install boto3 langfuse awscli --quiet

Authenticate the AWS session

import boto3

AWS_ACCESS_KEY_ID = "***"
AWS_SECRET_ACCESS_KEY = "***"
AWS_SESSION_TOKEN = "***"

bedrock = boto3.client(
    service_name="bedrock",
    region_name="eu-west-1",
    aws_access_key_id=AWS_ACCESS_KEY_ID,
    aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    aws_session_token=AWS_SESSION_TOKEN,
)

bedrock_runtime = boto3.client(
    service_name="bedrock-runtime",
    region_name="eu-west-1",
    aws_access_key_id=AWS_ACCESS_KEY_ID,
    aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    aws_session_token=AWS_SESSION_TOKEN,
)

Set Muster credentials

import os

os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
os.environ["LANGFUSE_BASE_URL"] = "https://app.getmuster.io"

from langfuse import get_client

langfuse = get_client()
if langfuse.auth_check():
    print("Muster client is authenticated and ready!")

Wrap the SDK

from langfuse import observe
from botocore.exceptions import ClientError

@observe(as_type="generation", name="Bedrock Converse")
def wrapped_bedrock_converse(**kwargs):
    kwargs_clone = kwargs.copy()
    input = kwargs_clone.pop("messages", None)
    modelId = kwargs_clone.pop("modelId", None)
    model_parameters = {
        **kwargs_clone.pop("inferenceConfig", {}),
        **kwargs_clone.pop("additionalModelRequestFields", {}),
    }
    langfuse.update_current_generation(
        input=input,
        model=modelId,
        model_parameters=model_parameters,
        metadata=kwargs_clone,
    )

    try:
        response = bedrock_runtime.converse(**kwargs)
    except (ClientError, Exception) as e:
        error_message = f"ERROR: Can't invoke '{modelId}'. Reason: {e}"
        langfuse.update_current_generation(level="ERROR", status_message=error_message)
        print(error_message)
        return

    response_text = response["output"]["message"]["content"][0]["text"]
    langfuse.update_current_generation(
        output=response_text,
        usage_details={
            "input": response["usage"]["inputTokens"],
            "output": response["usage"]["outputTokens"],
            "total": response["usage"]["totalTokens"],
        },
        metadata={"ResponseMetadata": response["ResponseMetadata"]},
    )

    return response_text

Run an example

conversation = [
    {"role": "user", "content": [{"text": "Hello! Recommend a finance book."}]}
]

@observe()
def run_examples():
    return {
        "anthropic": wrapped_bedrock_converse(
            modelId="eu.anthropic.claude-3-5-sonnet-20240620-v1:0",
            messages=conversation,
            inferenceConfig={"maxTokens": 500, "temperature": 1},
            additionalModelRequestFields={"top_k": 250},
        ),
        "llama3-2": wrapped_bedrock_converse(
            modelId="eu.meta.llama3-2-3b-instruct-v1:0",
            messages=conversation,
            inferenceConfig={"maxTokens": 500, "temperature": 1},
        ),
    }

run_examples()

Bedrock Converse trace in Muster

Cost & token tracking

Native framework integrations and the LiteLLM proxy report token usage to Muster automatically. With the @observe() decorator, report usage and cost directly via update_current_generation(usage_details=..., cost_details=...). Custom price information can be configured in the Muster UI.

Additional resources

AWS-maintained langfuse-genaiops notebook with AWS-specific examples.