LLM (OpenAI)

Step 1: Store your API key

1. Create an API key to access the OpenAI API, and save it in an environment variable.

   export OPENAI_API_KEY='<your-api-key>'

2. Create a Kubernetes Secret that stores the API key. By default, agentgateway reads the key from the Authorization field and sends it to the provider as a bearer token.

   kubectl apply -f- <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: openai-secret
     namespace: agentgateway-system
   type: Opaque
   stringData:
     Authorization: $OPENAI_API_KEY
   EOF

Step 2: Allow models on the Gateway

Allow the AgentgatewayModel route kind on the Gateway’s http listener. The listener accepts only the kinds you list in allowedRoutes.kinds, so keep HTTPRoute for the other quick starts.

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: agentgateway-proxy
  namespace: agentgateway-system
spec:
  gatewayClassName: agentgateway
  listeners:
  - protocol: HTTP
    port: 80
    name: http
    allowedRoutes:
      namespaces:
        from: All
      kinds:
      - group: gateway.networking.k8s.io
        kind: HTTPRoute
      - group: agentgateway.dev
        kind: AgentgatewayModel
EOF

Step 3: Expose the OpenAI models

Create an AgentgatewayModel resource that matches openai/* model names. Clients request names such as openai/gpt-5-mini. A model transformation strips the prefix, so OpenAI receives gpt-5-mini.

kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayModel
metadata:
  name: openai
  namespace: agentgateway-system
spec:
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: agentgateway-proxy
    sectionName: http
  match:
    model: "openai/*"
  provider: OpenAI
  policies:
    auth:
      secretRef:
        name: openai-secret
    transformations:
    - field: model
      expression: 'llmRequest.model.stripPrefix("openai/")'
EOF

Review the following table to understand this configuration. For more information, see the API docs.

Step 4: Send a request

1. Get the address of the agentgateway proxy.

   Cloud Provider LoadBalancerPort-forward for local testing

   Tip

   Kind cluster? Kind does not support LoadBalancer services by default. To use this option with a Kind cluster, install and run cloud-provider-kind.

   export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
   echo $INGRESS_GW_ADDRESS

   Use port-forwarding if your cluster does not assign an external IP to LoadBalancer services, as in a default Kind cluster.

   kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 8080:80

   In a separate terminal, point the requests in this guide at the forwarded port.

   export INGRESS_GW_ADDRESS=localhost:8080

2. Send a chat completion request for an OpenAI model. Prefix the model name with openai/.

   curl -s "http://$INGRESS_GW_ADDRESS/v1/chat/completions" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "openai/gpt-5-mini",
       "messages": [{"role": "user", "content": "Say hello in one sentence."}]
     }' | jq

   The response uses the provider’s model name without the openai/ prefix.

   Example output (abbreviated):

   {
     "model": "gpt-5-mini-2025-08-07",
     "object": "chat.completion",
     "choices": [
       {
         "index": 0,
         "message": {
           "role": "assistant",
           "content": "Hello! How can I help you today?"
         },
         "finish_reason": "stop"
       }
     ],
     "usage": {
       "prompt_tokens": 13,
       "completion_tokens": 18,
       "total_tokens": 31
     }
   }

3. Optional: List the models that clients can request. Agentgateway expands openai/* into the OpenAI models from its built-in model catalog.

   curl -s "http://$INGRESS_GW_ADDRESS/v1/models" | jq -r '.data[].id'

   Example output (abbreviated):

   openai/gpt-4.1
   openai/gpt-4o
   openai/gpt-5
   openai/gpt-5-mini
   ...