Ollama

Separate the local runtime from Cloud

Ollama is not a single model vendor. It provides a local runtime, an official model library, and the Ollama Cloud API. The native local base URL is http://localhost:11434/api; Cloud uses https://ollama.com/api. This site's /v1/chat/completions is a gateway compatibility route, so neither provider URL should be presented as a route on this site. Available tags, licenses, templates, quantization, and capabilities come from the exact Library record.

Authentication depends on where the call goes

The default localhost API requires no authentication. A direct Ollama Cloud call must send Authorization: Bearer <OLLAMA_API_KEY>. Some OpenAI SDKs require an api_key field even when pointed at the local compatibility endpoint, but local Ollama ignores that value. Do not interchange an Ollama Cloud key, this site's key, and a local SDK placeholder.

Native responses stream NDJSON by default

Native generation endpoints such as /api/generate default to multiple application/x-ndjson JSON lines, not one complete JSON document. Parse one line at a time until the completion marker; set "stream": false when a single JSON response is required. Even after the stream starts with HTTP 200, a later line can contain an error object, so inspect every record instead of trusting only the initial status.

Budget context and memory explicitly

Ollama's default context length is 4096 tokens. Change the service default with OLLAMA_CONTEXT_LENGTH or set num_ctx in a request. Longer context consumes more RAM or VRAM and reduces practical concurrency, so benchmark with real prompts, retrieved passages, and output budgets rather than relying on a model's theoretical limit.

Do not expose the local service unprotected

Ollama binds to 127.0.0.1:11434 by default. If OLLAMA_HOST is changed to a non-loopback listener, treat it as a service without built-in local authentication: restrict sources with a firewall or private network and add TLS, authentication, rate limits, and request-size limits at a reverse proxy. Do not publish management endpoints or local-file capabilities directly to the internet.

Errors, cost, and model safety

For 400, inspect request fields; for 404, check whether a model is pulled or available in Cloud; 429 is rate limiting; 500/502 can indicate service or upstream failure. Retry only transient failures with bounds. Cloud calls have quota and monetary cost, while local operation still costs downloads, disk, RAM, GPU capacity, and concurrency. Verify provenance, digest, license, template, and safety evaluation for community models before production.