Skip to main content
You built a LangGraph agent that works in a notebook. Now you need it behind an API with authentication, conversation memory, guardrails, and tracing. This guide takes you from a working StateGraph to a production endpoint in under 5 minutes.

The production gap

LangGraph gives you a graph-based runtime for building agents. It does not give you the infrastructure to serve them. When you move from graph.invoke() in a script to handling real traffic, you run into five missing pieces:

No API layer

No built-in HTTP server. You write FastAPI routes, CORS, request parsing, and streaming yourself. LangServe is deprecated. LangGraph Platform requires a LangSmith account.

No conversation memory

LangGraph supports checkpointers, but you wire up database connections, async lifecycle, and thread ID routing from HTTP requests yourself.

No guardrails

Your agent will process PII, jailbreak attempts, and toxic content unless you build input/output validation from scratch.

No observability

When a production agent returns garbage at 3am, you need traces. LangGraph has no built-in tracing. You instrument it yourself.
And one more: no streaming protocol. Modern chat UIs expect Server-Sent Events with structured events (text deltas, tool calls, thinking indicators). LangGraph emits raw astream_events that you need to map to a protocol your frontend understands. Idun Agent Platform fills all five gaps. Your StateGraph stays unchanged. Idun wraps it into a FastAPI service with AG-UI streaming, configurable memory, guardrails, and multi-provider observability. Configure everything through a YAML file or through the Manager UI.

What you will build

1

Streaming API

A LangGraph agent served as a REST + AG-UI streaming endpoint.
2

Conversation memory

In-memory persistence, upgradeable to PostgreSQL or SQLite.
3

Input guardrails

PII detection that blocks requests before they reach the agent.
4

Observability

Langfuse tracing on every invocation with full LLM call details.

Prerequisites

  • Python 3.12+
  • A Gemini API key (or any LangChain-compatible LLM)
  • 5 minutes
For the Manager UI path, you also need Docker and Docker Compose.

Choose your path

Install the engine SDK, write a YAML config, and run the agent from the CLI. Best for developers who want full control and minimal dependencies.
Both paths produce the same result: a production-ready LangGraph API.

Step 1: Write your LangGraph agent

This step is the same regardless of which path you choose. Create a project directory with an agent that has tool calling built in:
mkdir langgraph-prod && cd langgraph-prod
mkdir agent && touch agent/__init__.py
agent/agent.py
import os

from langchain_core.tools import tool
from langchain_google_genai import ChatGoogleGenerativeAI
from langgraph.graph import MessagesState, START, StateGraph
from langgraph.prebuilt import tools_condition


@tool
def add(a: float, b: float) -> float:
    """Add two numbers together."""
    return a + b


@tool
def multiply(a: float, b: float) -> float:
    """Multiply two numbers together."""
    return a * b


model = ChatGoogleGenerativeAI(
    model="gemini-2.5-flash",
    google_api_key=os.getenv("GEMINI_API_KEY"),
)

tools = [add, multiply]


async def call_model(state: MessagesState):
    model_with_tools = model.bind_tools(tools)
    response = await model_with_tools.ainvoke(state["messages"])
    return {"messages": [response]}


async def call_tools(state: MessagesState):
    from langchain_core.messages import ToolMessage

    tools_by_name = {t.name: t for t in tools}
    last_message = state["messages"][-1]
    results = []
    for tool_call in last_message.tool_calls:
        t = tools_by_name[tool_call["name"]]
        result = await t.ainvoke(tool_call["args"])
        results.append(
            ToolMessage(
                content=str(result),
                tool_call_id=tool_call["id"],
                name=tool_call["name"],
            )
        )
    return {"messages": results}


workflow = StateGraph(MessagesState)
workflow.add_node("call_model", call_model)
workflow.add_node("tools", call_tools)
workflow.add_edge(START, "call_model")
workflow.add_conditional_edges("call_model", tools_condition)
workflow.add_edge("tools", "call_model")
Two things matter here:
  1. The variable workflow is an uncompiled StateGraph. Do not call .compile(). Idun compiles it for you with the configured checkpointer and store.
  2. The state uses MessagesState (a TypedDict with a single messages field). Idun auto-detects this as a chat-mode agent. If your state has additional fields, Idun treats it as a structured-input agent and exposes the full JSON Schema through the capabilities endpoint.

Step 2: Set up Idun and run the agent

Install the engine SDK:
pip install idun-agent-engine langchain-google-genai
Create config.yaml next to your agent directory:
config.yaml
server:
  api:
    port: 8800

agent:
  type: LANGGRAPH
  config:
    name: "Production Chatbot"
    graph_definition: "./agent/agent.py:workflow"
    checkpointer:
      type: memory
That is a complete, valid config. Three fields define your entire setup:
  • type: LANGGRAPH tells Idun which adapter to use.
  • graph_definition: "./agent/agent.py:workflow" points to your file and variable. Format: path/to/file.py:variable_name. Idun dynamically imports it.
  • checkpointer.type: memory enables in-memory conversation persistence. Every request with the same thread_id continues the conversation.
Start the server:
export GEMINI_API_KEY="your-gemini-api-key"
idun agent serve --source file --path config.yaml
You should see:
Agent 'Production Chatbot' initialized and ready to serve!
Starting Idun Agent Engine server on http://localhost:8800...
Your agent is now live. Open http://localhost:8800/docs to see the full OpenAPI spec.

Step 3: Test the agent

The canonical endpoint is POST /agent/run, which implements the AG-UI streaming protocol used by CopilotKit, Vercel AI SDK, and other modern chat frontends.
curl -X POST http://localhost:8800/agent/run \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{
    "thread_id": "session-1",
    "messages": [
      {"id": "msg-1", "role": "user", "content": "What is 25 multiplied by 17?"}
    ]
  }'
You get back a stream of AG-UI events: RunStarted, TextMessageStart, ToolCallStart, ToolCallEnd, TextMessageContent (with deltas), TextMessageEnd, RunFinished. Your frontend renders these as they arrive.You can also check capabilities and health:
curl http://localhost:8800/agent/capabilities
curl http://localhost:8800/health

Step 4: Add your first guardrail

Guardrails validate input and output at the API boundary. They run before and after your agent, blocking requests that violate your policies. Idun uses Guardrails AI validators, downloaded and run locally.
Add a guardrails section to your config.yaml:
config.yaml
server:
  api:
    port: 8800

agent:
  type: LANGGRAPH
  config:
    name: "Production Chatbot"
    graph_definition: "./agent/agent.py:workflow"
    checkpointer:
      type: memory

guardrails:
  input:
    - config_id: detect_pii
      pii_entities: ["EMAIL_ADDRESS", "PHONE_NUMBER", "CREDIT_CARD"]
    - config_id: ban_list
      banned_words: ["ignore previous instructions", "system prompt"]
Set your Guardrails AI API key and restart:
export GUARDRAILS_API_KEY="your-guardrails-api-key"
idun agent serve --source file --path config.yaml
On startup, Idun downloads the guardrail validators from the Guardrails AI Hub and initializes them locally. The first start takes 30-60 seconds while models download. Subsequent starts are instant. Test it by sending a message that contains PII:
curl -X POST http://localhost:8800/agent/run \
  -H "Content-Type: application/json" \
  -d '{
    "thread_id": "session-2",
    "messages": [
      {"id": "msg-1", "role": "user", "content": "My email is john@company.com and my phone is 555-0123"}
    ]
  }'
The request is blocked with HTTP 429 before it ever reaches your agent. The PII guardrail detected an email address and phone number in the input. Idun supports 15 guardrail types: PII detection, ban lists, toxic language, jailbreak detection, prompt injection, NSFW filtering, bias detection, topic restriction, gibberish detection, competition checks, language validation, code scanning, RAG hallucination detection, Google Model Armor, and custom LLM-based validation.

Step 5: Add observability with Langfuse

Add an observability section to your config.yaml:
observability:
  - provider: LANGFUSE
    enabled: true
    config:
      host: "https://cloud.langfuse.com"
      public_key: "pk-lf-your-public-key"
      secret_key: "sk-lf-your-secret-key"
      run_name: "production-chatbot"
Restart the server. Every invocation now shows up in your Langfuse dashboard with full traces.
Every invocation now shows up in your Langfuse dashboard with full traces: LLM calls, token counts, latencies, tool executions. Langfuse trace showing LangGraph agent with tool calls Idun supports five observability providers that can run simultaneously:
ProviderMechanism
LangfuseLangChain callback handler
Arize PhoenixOpenTelemetry + OpenInference
LangSmithLangChain callback handler
GCP Cloud TraceOpenTelemetry span exporter
GCP Cloud LoggingPython logging integration
Add multiple providers by adding more entries to the observability list (or attaching multiple configs in the UI). They do not conflict.

Step 6: Upgrade to PostgreSQL memory

In-memory checkpointing loses conversations when the server restarts. For production, switch to PostgreSQL.
agent:
  type: LANGGRAPH
  config:
    name: "Production Chatbot"
    graph_definition: "./agent/agent.py:workflow"
    checkpointer:
      type: postgres
      db_url: "postgresql://postgres:postgres@localhost:5432/agent_memory"
Idun handles the async connection pool, table creation, and lifecycle management. Your agent code does not change.For a simpler persistence option, use SQLite:
    checkpointer:
      type: sqlite
      db_url: "sqlite:///conversations.db"
Your agent code does not change. Only the config does.

Step 7: Connect a frontend

The /agent/run endpoint implements the AG-UI protocol, which is natively supported by CopilotKit and compatible with any SSE-consuming frontend.
For standalone deployments, any frontend that speaks AG-UI or SSE can connect directly to http://your-host:8800/agent/run.

Full config.yaml

Here is the complete config file with everything from this guide: agent, guardrails, observability, and PostgreSQL memory. Copy it and adjust the values for your setup.
config.yaml
server:
  api:
    port: 8800

agent:
  type: LANGGRAPH
  config:
    name: "Production Chatbot"
    graph_definition: "./agent/agent.py:workflow"
    checkpointer:
      type: postgres
      db_url: "postgresql://postgres:postgres@localhost:5432/agent_memory"

guardrails:
  input:
    - config_id: detect_pii
      pii_entities: ["EMAIL_ADDRESS", "PHONE_NUMBER", "CREDIT_CARD"]
    - config_id: ban_list
      banned_words: ["ignore previous instructions", "system prompt"]

observability:
  - provider: LANGFUSE
    enabled: true
    config:
      host: "https://cloud.langfuse.com"
      public_key: "pk-lf-your-public-key"
      secret_key: "sk-lf-your-secret-key"
      run_name: "production-chatbot"

How Idun compares to alternatives

CapabilityManual FastAPILangServe (deprecated)LangGraph PlatformIdun Agent Platform
API servingYou build itProvidedProvided (cloud)Provided (self-hosted)
AG-UI streamingYou build itNot supportedNot supportedBuilt in
GuardrailsYou build itNot supportedNot supported15 types, YAML config
ObservabilityYou build itLimitedLangSmith only5 providers, simultaneous
Memory/checkpointingYou wire itLimitedBuilt inYAML config, 3 backends
MCP tool serversYou build itNot supportedNot supportedYAML config
SSO/OIDCYou build itNot supportedCloud-managedYAML config
Vendor lock-inNoneLangChain ecosystemLangSmith account requiredNone (open source, GPL-3.0)
Self-hostedYesYesNo (cloud only)Yes
Idun Agent Platform is open source and self-hosted. Your agent code stays yours. The config file is the only coupling, and it is a plain YAML file you can generate from any tool.

Next steps

You now have a production LangGraph agent with streaming, memory, guardrails, and observability. From here:
  • Add MCP tool servers to give your agent access to external tools via the Model Context Protocol. Add an mcp_servers section to your config or create one in the MCP page of the dashboard.
  • Add SSO/OIDC protection to require JWT authentication on all agent endpoints. Add an sso section with your OIDC issuer and client ID, or configure it through the SSO page.
  • Connect messaging integrations (WhatsApp, Discord, Slack) to expose your agent on external channels. Each is a config section with provider credentials, or a card in the Integrations page.
  • Manage prompts with versioning, Jinja2 variables, and per-agent assignment through the Prompts page.
The full configuration reference is at /configuration. The complete list of guardrail types is at /guardrails/reference.
Last modified on March 31, 2026