BYOKchat
BYOKchat Blog

How to Run a Private AI API Over Tailscale

Expose Ollama, LM Studio, or another local AI server privately across devices using Tailscale networking, Serve, HTTPS, access controls, authentication, and safe endpoint design.

· 6 min read

On this page
  1. Tailscale does not require the AI server to become public
  2. Two common patterns
  3. Pattern 1: Connect directly to the server’s Tailscale address
  4. Pattern 2: Keep AI server on localhost and use Tailscale Serve
  5. Serve is private; Funnel is a different scope
  6. HTTPS is useful even inside the private network
  7. Access control rules still matter
  8. Keep API authentication as another layer
  9. A simple Serve workflow
  10. Why Serve can be cleaner than binding to all interfaces
  11. The client base URL should use the Serve name
  12. Path prefixes need intentional mapping
  13. Tailscale does not fix a stopped model server
  14. Sleep still matters
  15. Mobile network transitions are easier to survive conceptually
  16. Do not store tailnet identity as a secret
  17. Use HTTPS certificate validation normally
  18. Think about prompt/tool data beyond the model request
  19. Apply least privilege to access controls
  20. Rate limiting still helps
  21. Prefer private overlay over public port forwarding
  22. Public port forwarding
  23. Private tailnet
  24. Test failure cases
  25. Where BYOKchat fits
  26. Further reading

A local AI server is easy to reach at home and awkward to reach when your phone leaves the Wi-Fi network.

The unsafe shortcut is router port forwarding:

public internet → home router → AI server port

A safer personal setup is a private overlay network such as Tailscale.

The goal becomes:

Diagram illustrating the surrounding section

Only devices/users allowed into the private network can reach the service.

Tailscale does not require the AI server to become public

Tailscale assigns private connectivity between devices in a tailnet.

Your Mac can continue running a model service locally while the iPhone reaches the Mac through its Tailscale identity.

This avoids exposing the inference port through the home router.

Two common patterns

Pattern 1: Connect directly to the server’s Tailscale address

Configure the AI server to listen on an interface reachable through Tailscale, then connect to the Mac’s tailnet IP or MagicDNS name.

Conceptually:

http://mac-name:1234/v1

inside the tailnet.

This is simple, but the service must listen on an address Tailscale traffic can reach and you still need to think about application authentication and HTTP/TLS.

Pattern 2: Keep AI server on localhost and use Tailscale Serve

Tailscale Serve can reverse proxy a local service and expose it to other tailnet members.

Conceptually:

AI runtime: http://127.0.0.1:1234
        ↑
Tailscale Serve reverse proxy
        ↑
https://mac-name.tailnet.ts.net
        ↑
iPhone in tailnet

This lets the AI runtime remain loopback-only.

Serve is private; Funnel is a different scope

Tailscale documentation distinguishes Serve from Funnel.

Serve makes the service available inside the tailnet.

Funnel is designed for public internet exposure.

For a private personal AI API, verify you are using the private Serve configuration and have not accidentally enabled a public exposure path.

HTTPS is useful even inside the private network

Tailscale Serve currently supports HTTPS and can provision TLS certificates for tailnet DNS names when the tailnet has HTTPS enabled.

That gives the client a normal HTTPS endpoint:

https://mac-name.<tailnet>.ts.net

The local AI runtime can still be plain HTTP on loopback behind the proxy.

Access control rules still matter

A tailnet can contain multiple users/devices.

Do not assume membership means every device should reach every AI service.

Use Tailscale access-control policy to restrict:

which users/devices
→ can connect to which service/port

This creates a network authorization layer before the request reaches the AI server.

Keep API authentication as another layer

Network membership and application authentication solve different problems.

A robust setup can use both:

Tailscale ACL
+ HTTPS
+ AI API token

If the local runtime supports an API token, enable it.

If not, the reverse proxy can potentially enforce another authentication mechanism.

Defense in depth is especially valuable when the tailnet has more than one person/device.

A simple Serve workflow

Exact CLI syntax evolves, so use current Tailscale documentation, but the conceptual steps are:

  1. install/sign in to Tailscale on the Mac;
  2. install/sign in on the iPhone;
  3. verify both devices appear in the same tailnet;
  4. run the local AI server on the Mac;
  5. verify the AI server works on Mac localhost;
  6. configure Tailscale Serve to reverse proxy that local port;
  7. connect from the iPhone using the private HTTPS name;
  8. apply access controls and API authentication.

Do not skip local validation before adding the overlay layer.

Why Serve can be cleaner than binding to all interfaces

Without a proxy, a developer may configure:

AI server listens on 0.0.0.0:1234

That can expose it to the entire LAN interface set.

With a private reverse proxy, the runtime can remain:

127.0.0.1:1234

and only the proxy is reachable remotely.

That reduces the inference server’s direct exposure surface.

The client base URL should use the Serve name

If Serve exposes:

https://mac-name.example.ts.net

and proxies the AI server root, an OpenAI-compatible base URL might be:

https://mac-name.example.ts.net/v1

depending on how the proxy path is configured.

The client should show its resolved endpoints so path mapping mistakes are obvious.

Path prefixes need intentional mapping

Suppose the local AI server expects:

/v1/chat/completions

If the proxy mounts the service under:

/ai/

then the client may need:

https://mac-name.example.ts.net/ai/v1

Do not assume proxy path rewriting.

Test /models or another harmless endpoint first.

Tailscale does not fix a stopped model server

The overlay can be healthy while the AI runtime is offline.

Diagnose separately:

Can iPhone reach Mac over Tailscale?
Can Serve reach localhost target?
Is AI server running?
Is model loaded/available?

A 502-like proxy failure is different from tailnet connectivity failure.

Sleep still matters

If the Mac sleeps deeply, remote inference may become unavailable.

For reliable remote access, consider:

  • appropriate power settings;
  • an always-on home server;
  • auto-starting the AI runtime;
  • Wake-on-LAN/remote-wake behavior where suitable.

The client should still handle offline state gracefully.

Mobile network transitions are easier to survive conceptually

The iPhone can move from home Wi-Fi to cellular while retaining Tailscale connectivity.

Your HTTP stream may still disconnect during interface transitions.

The AI client should persist partial output and support recovery/cancellation regardless of the overlay network.

See How to Resume or Recover an Interrupted AI Generation.

Do not store tailnet identity as a secret

The hostname itself usually is not an authentication secret.

Credentials still belong in secure storage:

  • AI API token;
  • proxy-specific secret headers;
  • other service credentials.

A configuration export can include the private base URL while omitting tokens.

Use HTTPS certificate validation normally

One benefit of a correctly configured Serve HTTPS endpoint is that clients can use normal TLS validation for the tailnet DNS name.

Do not add certificate-verification bypasses just because the endpoint is private.

If TLS validation fails, debug DNS/certificate configuration instead.

Think about prompt/tool data beyond the model request

A private network protects transport routing, but AI workflows may still call cloud tools or MCP servers.

A “local model over Tailscale” conversation can still send data elsewhere if:

  • web search is enabled;
  • remote MCP tools are enabled;
  • attachments are uploaded to another service;
  • telemetry includes content.

Communicate those boundaries separately.

Apply least privilege to access controls

If only the user’s iPhone and Mac need the AI server, do not grant every tailnet device access to the port.

Network policy should express the actual requirement.

For teams, consider a dedicated service identity/ACL rather than broad device-to-device connectivity.

Rate limiting still helps

Private does not mean infinite capacity.

A local GPU can be exhausted by a few concurrent long generations.

Use:

  • client concurrency limits;
  • API/proxy rate limits where appropriate;
  • cancellation;
  • bounded retries.

This also limits accidental loops from agent/tool workflows.

Prefer private overlay over public port forwarding

Compare the security posture:

Public port forwarding

internet can scan port
must harden public edge
TLS/auth/rate limiting mandatory
home IP/DNS exposure

Private tailnet

service reachable only through private network membership/policy
no public router forwarding required
stable private names
optional private HTTPS reverse proxy

For a personal BYOK setup, the latter is usually easier to reason about.

Test failure cases

Before relying on remote local AI, test:

iPhone on home Wi-Fi
iPhone on cellular
Tailscale disconnected on phone
Tailscale disconnected on Mac
AI server stopped
Serve proxy target wrong
API token wrong
ACL denies access
Mac asleep
stream interrupted during network transition

A setup that only works under one ideal network condition will create support problems later.

Where BYOKchat fits

A BYOK client can treat a Tailscale-served endpoint as a normal custom HTTPS connection. The app does not need Tailscale-specific protocol code; it only needs a stable base URL, credentials, model discovery/manual model support, and clear diagnostics.

That is a good example of keeping networking infrastructure outside the provider abstraction.

Further reading

Keep reading