Skip to content

Blog

Smart LLM API Routing with GOST Reverse Proxy

AI is getting increasingly popular, with more and more LLMs and providers emerging. But tokens remain a finite resource, and single-provider limits are easily reached. Multi-provider combinations with dynamic model-based routing are becoming a more cost-effective approach.

Protocol Conversion and Model Routing

Most mainstream model APIs fall into a few categories:

  • OpenAI Chat — /v1/chat/completions, the default API format for many providers.
  • OpenAI Responses — /v1/responses, currently used by Codex.
  • Anthropic Messages — /v1/messages, currently used by Claude Code.

The challenge becomes connecting clients like Claude Code and Codex to various providers that support these compatible APIs.

Protocol Conversion

Different providers offer different APIs. For example, DeepSeek's official API provides OpenAI Chat and Anthropic Messages compatibility, but not OpenAI Responses — so it can't be used with Codex directly. OpenCode-Go's DeepSeek only provides an OpenAI Chat interface, which neither Claude Code nor Codex can use directly. This mismatch is the biggest obstacle to model routing.

If we set aside the complexity of protocol conversion, at the API level a model API call is simply a standard HTTP request. From this perspective, LLM API routing becomes pure HTTP protocol conversion — exactly what a reverse proxy does.

HTTP protocol conversion has three layers:

Host and URI Rewriting

Clients typically use standard URIs — Claude Code uses /v1/messages, Codex uses /v1/responses. But providers may use different URIs — DeepSeek uses /anthropic/v1/messages, OpenCode-Go uses /zen/go/v1/chat/completions. The reverse proxy rewrites the Host and URI based on the target provider's API endpoint.

Request Header Modification

API calls require API key authentication. When combining multiple providers, the client can't specify the API key directly. Since authentication is in the HTTP Authorization header, the reverse proxy must set the appropriate API key for each provider.

Request/Response Body Transformation

This is the most complex and important part — protocol conversion between different API formats, such as converting an Anthropic Messages request to an OpenAI Chat request and converting the response back.

All three conversion layers are completely transparent to the client — the client simply points its model URL to the reverse proxy.

Model Routing

Clients like Claude Code and Codex typically provide model switching. Claude Code currently offers Fable, Opus, Sonnet, and Haiku models. Codex offers gpt-5.5, gpt-5.4, gpt-5.3, gpt-5.2. The common approach is to configure model mappings on the client side. For more flexibility, the reverse proxy can handle model selection dynamically.

For example, mapping the model field in the request body: claude-fable → OpenCode-Go's glm-5.2, claude-opus → DeepSeek's deepseek-v4-pro, etc. Adjusting mappings only requires changing the reverse proxy configuration — transparent to clients.

GOST Reverse Proxy

Unlike dedicated LLM gateways, GOST's reverse proxy is a general-purpose port forwarding service that doesn't couple with any specific business logic (LLM routing). It provides the following features:

Host and URI Rewriting

hops:
- name: hop-0
  nodes:
  - name: opencode-go-deepseek
    addr: opencode.ai:443
    http:
      host: opencode.ai
      rewriteURL:
      - match: /v1/messages
        replacement: /zen/v1/chat/completions
      - match: /v1/responses
        replacement: /zen/v1/chat/completions

Custom Request Headers

hops:
- name: hop-0
  nodes:
  - name: opencode-go-deepseek
    addr: opencode.ai:443
    http:
      requestHeader:
        Authorization: "Bearer your-opencode-api-key"

Request/Response Body Rewriting

GOST provides a Rewriter plugin for rewriting request/response bodies, implemented externally via plugins:

hops:
- name: hop-0
  nodes:
  - name: opencode-go-deepseek
    addr: opencode.ai:443
    http:
      rewriteRequestBody:
      - rewriter: openai-converter
      rewriteResponseBody:
      - rewriter: openai-converter

rewriters:
- name: openai-converter
  plugin:
    type: http
    addr: http://localhost:8000/rewrite

GOST provides an LLM API conversion Rewriter plugin service — llm-api-converter:

./llm-api-converter \
  --addr :8000 \
  --model deepseek-v4-flash \
  --model-map "claude-fable=glm-5.2:openai,claude-opus=deepseek-v4-pro:openai,*=deepseek-v4-flash:openai"

Request Body Matching

Nodes can use matching rules to select providers:

hops:
- name: hop-0
  nodes:
  - name: opencode-go-deepseek
    addr: opencode.ai:443
    matcher:
      rule: '(BodyRegexp(`"model"\s*:\s*"claude-opus[^"]*"`) || BodyRegexp(`"model"\s*:\s*"gpt-5.4[^"]*"`))'
      bodySize: 65536

Complete Example

Integration with OpenCode-Go and OpenCode-Zen:

services:
  - name: llm-proxy
    addr: :8787
    handler:
      type: tcp
      metadata:
        sniffing: true
    listener:
      type: tcp
    forwarder:
      hop: hop-0

hops:
  - name: hop-0
    nodes:
      - name: opencode-go-glm-5.2
        addr: opencode.ai:443
        matcher:
          rule: 'Method(`POST`) && Header(`Content-Type`, `application/json`) && (BodyRegexp(`"model"\s*:\s*"claude-fable[^"]*"`) || BodyRegexp(`"model"\s*:\s*"gpt-5.5[^"]*"`))'
          bodySize: 65536
        tls:
          secure: true
          serverName: opencode.ai
        http:
          host: opencode.ai
          rewriteURL:
            - match: /v1/messages
              replacement: /zen/go/v1/chat/completions
            - match: /v1/responses
              replacement: /zen/go/v1/chat/completions
          requestHeader:
            Authorization: "Bearer your-opencode-apikey"
          rewriteRequestBody:
            - rewriter: openai-converter
          rewriteResponseBody:
            - rewriter: openai-converter

      - name: opencode-go-deepseek-v4-pro
        addr: opencode.ai:443
        matcher:
          rule: 'Method(`POST`) && Header(`Content-Type`, `application/json`) && (BodyRegexp(`"model"\s*:\s*"claude-opus[^"]*"`) || BodyRegexp(`"model"\s*:\s*"gpt-5.4[^"]*"`))'
          bodySize: 65536
        tls:
          secure: true
          serverName: opencode.ai
        http:
          host: opencode.ai
          rewriteURL:
            - match: /v1/messages
              replacement: /zen/go/v1/chat/completions
            - match: /v1/responses
              replacement: /zen/go/v1/chat/completions
          requestHeader:
            Authorization: "Bearer your-opencode-apikey"
          rewriteRequestBody:
            - rewriter: openai-converter
          rewriteResponseBody:
            - rewriter: openai-converter

      - name: opencode-zen-deepseek-v4-flash-free
        addr: opencode.ai:443
        matcher:
          rule: 'Method(`HEAD`) || Method(`POST`) && Header(`Content-Type`, `application/json`)'
        tls:
          secure: true
          serverName: opencode.ai
        http:
          host: opencode.ai
          rewriteURL:
            - match: /v1/messages
              replacement: /zen/v1/chat/completions
            - match: /v1/responses
              replacement: /zen/v1/chat/completions
            - match: /
              replacement: /zen
          requestHeader:
            Authorization: "Bearer your-opencode-apikey"
          rewriteRequestBody:
            - rewriter: openai-converter
          rewriteResponseBody:
            - rewriter: openai-converter

rewriters:
  - name: openai-converter
    plugin:
      type: http
      addr: http://localhost:8000/rewrite

Start the services:

./gost -C gost.yaml
./llm-api-converter \
  --addr :8000 \
  --model-map "gpt-5.5=glm-5.2:openai,claude-fable=glm-5.2:openai,gpt-5.4=deepseek-v4-pro:openai,claude-opus=deepseek-v4-pro:openai,gpt-5.3=deepseek-v4-flash:openai,claude-sonnet=deepseek-v4-flash:openai,*=deepseek-v4-flash-free:openai"

Data Flow

Anthropic Messages client (Claude Code): 

POST :8787/v1/messages (model: claude-opus-4-8)
  → Sniffer reads HTTP Body prefix
  → BodyRegexp matches "claude-opus" → selects opencode-go-deepseek-v4-pro
  → rewriteURL: /v1/messages → /zen/go/v1/chat/completions
  → rewriteRequestBody: convert protocol, replace model with deepseek-v4-pro
  → Forward to https://opencode.ai/zen/go/v1/chat/completions

OpenAI Responses client (Codex): 

POST :8787/v1/responses (model: gpt-5.5)
  → Sniffer reads HTTP Body prefix
  → BodyRegexp matches "gpt-5.5" → selects opencode-go-glm-5.2
  → rewriteURL: /v1/responses → /zen/go/v1/chat/completions
  → rewriteRequestBody: convert protocol, replace model with glm-5.2
  → Forward to https://opencode.ai/zen/go/v1/chat/completions

Catch-all (other requests): 

POST :8787/v1/messages (model: claude-haiku-5)
  → Sniffer reads HTTP Body prefix
  → Catch-all rule → selects opencode-zen-deepseek-v4-flash-free
  → rewriteURL: /v1/messages → /zen/v1/chat/completions
  → rewriteRequestBody: convert protocol, replace model with deepseek-v4-flash-free
  → Forward to https://opencode.ai/zen/v1/chat/completions

Docker Compose

services:
  gost:
    image: gogost/gost
    ports:
      - "8787:8787"
    volumes:
      - ./gost.yaml:/etc/gost/gost.yaml
    depends_on:
      - llm-converter

  llm-converter:
    image: ginuerzh/llm-api-converter
    command:
      - --addr=:8000
      - --model-map=gpt-5.5=glm-5.2:openai,claude-fable=glm-5.2:openai,...
    ports:
      - "8000:8000"

Note: In Docker Compose, services communicate by container name, so change the rewriter address to http://llm-converter:8000/rewrite.

Start:

docker compose up -d
  • in Logging
  • 10 min read

Logging in GOST

Program logs are valuable for both developers and users. For developers, they help quickly locate issues. For users, logs can be shared with developers for problem analysis, and can also be used for usage statistics and analysis. Logs are also a core component of observability.

User-Level Traffic Statistics and Dynamic Rate Limiting

GOST's Observer component collects connection and traffic statistics. When configured, it periodically reports total input/output bytes as events. The Limiter component enforces connection and traffic limits.

Sometimes finer-grained traffic management is needed. For example, an authenticated proxy service may require per-user traffic statistics or rate limiting, possibly with dynamic adjustments based on real-time usage. Since different scenarios may have complex logic, GOST doesn't provide built-in user-level limiting — instead, it exposes a plugin interface for custom implementations.

Local Debugging of K8s Services Using Reverse Proxy Tunnel

Cloud-native technologies like containers and Kubernetes make service deployment and management more flexible and convenient. However, debugging applications running in a Kubernetes cluster can be challenging.

Tools like Telepresence solve this by intercepting service traffic and forwarding it to a local service. We can achieve similar functionality using the reverse proxy tunnel.

GOST.PLUS — Public Reverse Proxy Service

The reverse proxy tunnel is one of the major new features in GOST, and a very important one. With reverse proxy and intranet penetration, you can easily expose internal web services to the public network for access anytime, anywhere.

To test this feature more comprehensively and provide a quick way for users needing temporary public access to internal services, we launched the GOST.PLUS public reverse proxy test service. This service is open to all users without registration.

  • in Serial
  • 15 min read

Serial Port Redirector

Serial ports are largely absent from modern personal computers, but can still be found on industrial devices and embedded systems. Since serial communication differs significantly from TCP/IP, debugging and analyzing serial-based protocols requires different approaches.

GOST added serial port redirector functionality after v3.0.0-rc8. This enables forwarding local serial port data to a TCP service, or TCP service data to a local serial port, or even forwarding between remote serial ports. Serial forwarding enables two use cases: remote serial communication and serial data monitoring.

Reverse Proxy Tunnel in Practice

The previous post introduced reverse proxy and intranet penetration concepts. This post demonstrates practical use of the reverse proxy tunnel through concrete examples.

A reverse proxy tunnel combines reverse proxy with intranet penetration. These two concepts aren't inherently linked — reverse proxy can function without intranet penetration, and intranet penetration isn't solely for reverse proxy. However, many scenarios require combining them. For example, home or corporate networks may lack a public IP, making direct public access impossible — intranet penetration via a public IP server provides indirect access to intranet services.

Reverse Proxy and Intranet Penetration

Reverse proxy is a type of proxy service. In common proxy services like HTTP/SOCKS5 proxies, the proxy targets the client — the proxy acts on behalf of the client to connect to the target server. In reverse proxy, the proxy targets the server. This is why HTTP/SOCKS5 proxies are also called forward proxies. The main difference is that in forward proxy, the client knows about the proxy, while in reverse proxy, the client (and even the server) may not know about the proxy — to the client, the proxy appears to be the actual service being accessed.

From a certain perspective, reverse proxy and port forwarding are similar — both create a mapping between two ports, forwarding data from one to another. However, reverse proxy uses the forwarded data for more precise control, while port forwarding typically doesn't depend on data content and is purely end-to-end forwarding. GOST's reverse proxy is built on port forwarding.

  • in TUN, VPN
  • 30 min read

VPN Networking with TUN Devices

GOST first introduced TUN (and TAP) device support in v2.9. In v3 (beta.4), the implementation was changed from the songgao/water library to wireguard-go, with added heartbeat and authentication mechanisms.

The design philosophy for TUN devices in GOST is simplicity and lightness — no overly complex configuration, minimal data processing. If it meets specific use cases, that's sufficient. For more complex applications, WireGuard itself can be used directly.

TUN devices have many uses, most commonly for building VPNs. This post covers GOST-based TUN VPN networking.