Skip to content

Blog

Smart LLM API Routing with GOST Reverse Proxy

AI is getting increasingly popular, with more and more LLMs and providers emerging. But tokens remain a finite resource, and single-provider limits are easily reached. Multi-provider combinations with dynamic model-based routing are becoming a more cost-effective approach.

Logging in GOST

Program logs are valuable for both developers and users. For developers, they help quickly locate issues. For users, logs can be shared with developers for problem analysis, and can also be used for usage statistics and analysis. Logs are also a core component of observability.

User-Level Traffic Statistics and Dynamic Rate Limiting

GOST's Observer component collects connection and traffic statistics. When configured, it periodically reports total input/output bytes as events. The Limiter component enforces connection and traffic limits.

Sometimes finer-grained traffic management is needed. For example, an authenticated proxy service may require per-user traffic statistics or rate limiting, possibly with dynamic adjustments based on real-time usage. Since different scenarios may have complex logic, GOST doesn't provide built-in user-level limiting — instead, it exposes a plugin interface for custom implementations.

Local Debugging of K8s Services Using Reverse Proxy Tunnel

Cloud-native technologies like containers and Kubernetes make service deployment and management more flexible and convenient. However, debugging applications running in a Kubernetes cluster can be challenging.

Tools like Telepresence solve this by intercepting service traffic and forwarding it to a local service. We can achieve similar functionality using the reverse proxy tunnel.

GOST.PLUS — Public Reverse Proxy Service

The reverse proxy tunnel is one of the major new features in GOST, and a very important one. With reverse proxy and intranet penetration, you can easily expose internal web services to the public network for access anytime, anywhere.

To test this feature more comprehensively and provide a quick way for users needing temporary public access to internal services, we launched the GOST.PLUS public reverse proxy test service. This service is open to all users without registration.

Serial Port Redirector

Serial ports are largely absent from modern personal computers, but can still be found on industrial devices and embedded systems. Since serial communication differs significantly from TCP/IP, debugging and analyzing serial-based protocols requires different approaches.

GOST added serial port redirector functionality after v3.0.0-rc8. This enables forwarding local serial port data to a TCP service, or TCP service data to a local serial port, or even forwarding between remote serial ports. Serial forwarding enables two use cases: remote serial communication and serial data monitoring.

Reverse Proxy Tunnel in Practice

The previous post introduced reverse proxy and intranet penetration concepts. This post demonstrates practical use of the reverse proxy tunnel through concrete examples.

A reverse proxy tunnel combines reverse proxy with intranet penetration. These two concepts aren't inherently linked — reverse proxy can function without intranet penetration, and intranet penetration isn't solely for reverse proxy. However, many scenarios require combining them. For example, home or corporate networks may lack a public IP, making direct public access impossible — intranet penetration via a public IP server provides indirect access to intranet services.

Reverse Proxy and Intranet Penetration

Reverse proxy is a type of proxy service. In common proxy services like HTTP/SOCKS5 proxies, the proxy targets the client — the proxy acts on behalf of the client to connect to the target server. In reverse proxy, the proxy targets the server. This is why HTTP/SOCKS5 proxies are also called forward proxies. The main difference is that in forward proxy, the client knows about the proxy, while in reverse proxy, the client (and even the server) may not know about the proxy — to the client, the proxy appears to be the actual service being accessed.

From a certain perspective, reverse proxy and port forwarding are similar — both create a mapping between two ports, forwarding data from one to another. However, reverse proxy uses the forwarded data for more precise control, while port forwarding typically doesn't depend on data content and is purely end-to-end forwarding. GOST's reverse proxy is built on port forwarding.

VPN Networking with TUN Devices

GOST first introduced TUN (and TAP) device support in v2.9. In v3 (beta.4), the implementation was changed from the songgao/water library to wireguard-go, with added heartbeat and authentication mechanisms.

The design philosophy for TUN devices in GOST is simplicity and lightness — no overly complex configuration, minimal data processing. If it meets specific use cases, that's sufficient. For more complex applications, WireGuard itself can be used directly.

TUN devices have many uses, most commonly for building VPNs. This post covers GOST-based TUN VPN networking.