Local Gateway API

Call the EdgeSpeak API locally or from another trusted machine.

Quickstart

For local access, start the desktop app and its local gateway, then use the API key shown in the app. The desktop gateway binds to 127.0.0.1 and uses http://127.0.0.1:1117/v1 by default. Local authentication is off by default on fresh installs; turn it on in the app when you need it. Once it is on, the gateway accepts the Authorization: Bearer and x-api-key headers.

export EDGESPEAK_BASE_URL=http://127.0.0.1:1117/v1
export EDGESPEAK_API_KEY=sk-edgespeak-...

curl "$EDGESPEAK_BASE_URL/models" \
  -H "Authorization: Bearer $EDGESPEAK_API_KEY"

# Anthropic-compatible clients may use this instead:
# -H "x-api-key: $EDGESPEAK_API_KEY"

Call the API from another machine

Run the headless service on a machine with the required CPU or GPU resources and installed local models. The desktop app does not need to be installed or running. The desktop gateway accepts only local clients, while edgespeak-cli serve can explicitly listen on a LAN or private VPN address. By default, it still listens only on http://127.0.0.1:1118/v1.

On the service host (the machine running the service), load a long random key from protected secret storage, then start the service:

export EDGESPEAK_API_KEY="your-long-random-secret"
edgespeak-cli serve --host 0.0.0.0 --port 1118 --allow-remote

When listening on a non-loopback address, the service refuses to start unless you pass --allow-remote. It also requires a non-empty EDGESPEAK_API_KEY unless you explicitly pass --allow-unauthenticated. Consider disabling remote authentication only on an isolated test network.

On the client (the machine making the request), replace 192.168.1.50 with the service host's actual LAN or VPN address and use the same key:

export EDGESPEAK_BASE_URL=http://192.168.1.50:1118/v1
export EDGESPEAK_API_KEY="the-same-secret"

curl "$EDGESPEAK_BASE_URL/models" \
  -H "Authorization: Bearer $EDGESPEAK_API_KEY"

0.0.0.0 is only a server listening address; never put it in the client URL. When EdgeSpeak can identify one usable network interface, startup output includes an EdgeSpeak service network URL that you can copy. The service host handles model management and downloads, inference, and license validation. Uploaded audio, prompts, images, and returned responses travel between the two machines.

The headless service shares the gateway implementation with the desktop app: the paths and request shapes documented here are the same on both, and the base URL and the API key are what differ. Two inputs are desktop-only. Streaming transcription (stream=true) is refused with 400 on the headless listener. Local file paths are refused as well — the headless listener reads uploaded bytes, so send text instead of text_path on alignments, and inline segments[] instead of file on segmentations.

The headless service provides HTTP and does not include HTTPS. Use it only on a trusted LAN or private VPN, and use the service host's firewall to restrict TCP port 1118 to trusted sources. Alternatively, keep the default loopback listener and connect from another machine through an SSH tunnel:

ssh -L 1118:127.0.0.1:1118 user@edgespeak-host

If the client runs in a browser, add its exact page origin with --allow-origin https://your-app.example. If an HTTPS reverse proxy changes the Host header to a domain, allow the domain itself with --allow-host gateway.example. Clients that connect directly by IP address do not need --allow-host. See the CLI service guide for installation and startup instructions.

Audio and text

Transcribe uploaded media, align known text, segment text, synthesize speech, and manage reusable local voices. Transcription supports json, verbose_json, text, diarized_json, and JSON-mode SSE streaming; timestamp granularities are available in verbose and diarized output. The general request-body limit is 512 MiB.

POST   /v1/audio/transcriptions   # speech → text, JSON or SSE
POST   /v1/audio/alignments       # known text → word timing
POST   /v1/audio/speech           # text → speech
GET    /v1/audio/voices           # list voices
POST   /v1/audio/voices           # add a reusable voice
DELETE /v1/audio/voices/:voice_id # delete a user voice
POST   /v1/text/segmentations     # text or timed segments → sentences

Speakers

Use diarized_json for speaker-aware transcripts, or /v1/speaker/diarizations for an activity timeline without text. Speaker embeddings and cosine similarity are also available for local matching. num_speakers is optional clustering guidance, not an identity claim. Speaker endpoints accept uploaded bytes rather than arbitrary local paths.

POST /v1/audio/transcriptions
  response_format=diarized_json

POST /v1/speaker/diarizations
  multipart: file, num_speakers (optional)

POST /v1/speaker/embeddings
POST /v1/speaker/similarity

Realtime API

Connect over WebSocket for transcription sessions or full ASR → language model → speech sessions. Send session.update before audio, then append base64 PCM16LE mono frames. Server clients can authenticate with normal headers; browser WebSocket clients must offer the realtime and openai-insecure-api-key.<KEY> subprotocols. Local language models must be loaded before Realtime use.

const ws = new WebSocket(
  "ws://127.0.0.1:1117/v1/realtime",
  ["realtime", "openai-insecure-api-key.<KEY>"]
);

session.created
  → session.update              # type: transcription | realtime
  → input_audio_buffer.append   # base64 PCM16LE mono
  → input_audio_buffer.commit

events: conversation.item.input_audio_transcription.*
        response.output_text.*
        response.output_audio.*
        response.done | error

Language and vision APIs

Use the compatibility surface that matches your client: OpenAI Responses or Chat Completions, Anthropic Messages, or tokenization. Not every model supports every endpoint. Read supported_endpoints from /v1/models, and check execution_location: local models keep inputs on the device, while configured remote models receive the request sent to them.

POST /v1/responses          # OpenAI Responses-compatible
POST /v1/chat/completions   # OpenAI Chat Completions-compatible
POST /v1/messages           # Anthropic Messages-compatible
POST /v1/tokenize

curl http://127.0.0.1:1117/v1/responses \
  -H "Authorization: Bearer sk-edgespeak-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen/Qwen3.5-4B","input":"Summarize this meeting"}'

Native video input

POST /v1/chat/completions accepts input_video for installed local multimodal models on macOS and Linux builds with native video support. The desktop gateway and Headless service use the same model-specific preprocessing. Realtime WebSocket sessions do not accept input_video; image events there are still images, not a continuous video stream.

EdgeSpeak selects the sampling rate from the active model's catalog entry or imported-repository metadata. It is not a per-request field or a Server flag:

Local model family	Sampling rate	Reference preprocessing
Qwen 3.5 / Qwen 3.6	2 FPS	Qwen examples use `fps=2.0` with frame sampling enabled
Gemma 4	1 FPS	Gemma documents video capacity at 1 FPS

These values follow the model publishers' reference preprocessing rather than the source video's native frame rate. Switching model families restarts the local worker with the matching value. See the Qwen 3.5, Qwen 3.6, and Gemma 4 model documentation.

When the desktop app imports a GGUF repository from Hugging Face or ModelScope, it makes a best-effort read of root-level config.json, tokenizer_config.json, and processor_config.json. A detected video_processor.fps becomes the recommendation; known Qwen/Gemma families use the table above when metadata is absent; any other repository with an mmproj file defaults to 1 FPS. A vision declaration in config.json is shown as a warning unless a usable mmproj is also present. Users can override 1, 2, 4, or 8 FPS under Models → Generation settings. More sampled frames generally improve temporal coverage, but consume more visual tokens, context, memory, and processing time.

Encode the video file bytes as base64 in the Chat Completions content item:

{
  "model": "Qwen/Qwen3.5-4B",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Summarize this video in chronological order." },
        { "type": "input_video", "input_video": { "data": "<base64 MP4 bytes>" } }
      ]
    }
  ]
}

Video decoding requires both ffmpeg and ffprobe to be available on the service machine. EdgeSpeak does not bundle them; install the ffmpeg package with Homebrew on macOS or your Linux package manager. The complete JSON request is limited to 512 MiB, and base64 increases the source file size by roughly one third, so use short, appropriately sized clips.

Context window

Input text, conversation history, visual tokens produced from images or sampled video frames, and model output all share one context budget. A larger context can retain more history or video but allocates a larger KV cache in RAM or VRAM. A smaller context leaves more memory for the model and other workloads but reaches the request limit sooner.

In the desktop app, open Models → Generation settings → Context length on a local model card. The automatic value is currently 8K; explicit choices run from 4K up to that model's supported limit. Catalog models use their catalog limit. Hugging Face and ModelScope imports first inspect max_position_embeddings (including nested text/language config) or model_max_length, then use a known-family limit, and finally fall back to a 262K selectable ceiling when no declaration is available. The choice is saved per model. Saving it unloads the currently loaded local model, and the next request reloads the worker with the new context size.

Headless Server has an independent startup setting and does not read desktop preferences. Pass --context-tokens; the default is 8192 tokens, and the accepted range is 512–262144. Choose a value no larger than the active model's supported context. Changing it requires restarting that Headless Server process.

edgespeak-cli serve --context-tokens 32768

Model discovery and lifecycle

Use canonical IDs returned by /v1/models. Inspect running models separately, and explicitly load or unload local language models to control download state and memory. Model downloads can be started, polled for progress, and cancelled. A request may return 503 model_downloading with Retry-After while a required local model is being prepared.

GET  /v1/models
GET  /v1/models/running
POST /v1/models/load
POST /v1/models/unload

GET  /v1/models/downloads        # download progress
POST /v1/models/download         # start a download
POST /v1/models/download/cancel  # cancel a download

# Select by supported_endpoints and execution_location.
# Local language models must be loaded before Realtime use.

MCP

For agents, stdio through edgespeak-cli mcp is the recommended transport and can start the bundled local runtime without the app for supported tools. The desktop gateway and the headless service both expose Streamable HTTP at /mcp, each using its own authentication setting and API key.

edgespeak-cli mcp                   # recommended: stdio
POST http://127.0.0.1:1117/mcp      # Streamable HTTP, desktop gateway
POST http://127.0.0.1:1118/mcp      # Streamable HTTP, headless service

Errors, browser access, and security

License, quota, authentication, model-readiness, and busy-runtime failures use structured error responses. Non-browser clients without Origin are allowed, but Host is still checked. Browser requests are limited to loopback origins by default; explicitly allow a trusted non-loopback origin and proxy host with environment variables, and never expose the long-lived gateway key in public frontend code.

401  invalid or missing API key
403  host or origin is not allowed
413  request body exceeds 512 MiB
503  model_downloading              # honor Retry-After

EDGESPEAK_GATEWAY_PORT=1117
EDGESPEAK_GATEWAY_ALLOWED_ORIGINS=https://app.example.com
EDGESPEAK_GATEWAY_ALLOWED_HOSTS=proxy.example.com