Ollama

Run models locally with Ollama's OpenAI-compatible server, directly or through Yunxin.

Overview

Ollama is a tool for running and managing large language models locally. It ships an OpenAI-compatible server (default http://localhost:11434) and a simple CLI for pulling and running models.

Official Website: https://ollama.com Documentation: https://docs.ollama.com

Key features

Local deployment — Run models on your own machine
Simple CLI — Easy model pull and run
Model library — Browse models at ollama.com/library
OpenAI-compatible API — Standard chat completions endpoint, no key required locally
Custom models — Import and run GGUF models

Quick start

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run a model
ollama run model-id

Calling Ollama directly (local)

Point the OpenAI SDK at your local Ollama server. No API key is required, so any non-empty placeholder works. Note the OpenAI-compatible routes live under the /v1 path.

from openai import OpenAI

client = OpenAI(
    api_key="ollama",
    base_url="http://localhost:11434/v1"
)

response = client.chat.completions.create(
    model="model-id",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "ollama",
  baseURL: "http://localhost:11434/v1",
});

const response = await client.chat.completions.create({
  model: "model-id",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(response.choices[0].message.content);

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "model-id",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

If your Ollama instance is registered with Yunxin, address its models with the ollama/<model> prefix and use the standard Yunxin base URL (https://api.yuhuanstudio.com/v1) and your sk- API key. This lets you mix local models with hosted providers under one client.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["YUNXIN_API_KEY"],
    base_url="https://api.yuhuanstudio.com/v1"
)

response = client.chat.completions.create(
    model="ollama/model-id",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.YUNXIN_API_KEY,
  baseURL: "https://api.yuhuanstudio.com/v1",
});

const response = await client.chat.completions.create({
  model: "ollama/model-id",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(response.choices[0].message.content);

curl https://api.yuhuanstudio.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $YUNXIN_API_KEY" \
  -d '{
    "model": "ollama/model-id",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Available models

Ollama serves whichever models you have pulled. When accessed through Yunxin, query the Models API:

curl "https://api.yuhuanstudio.com/v1/models?provider=ollama" \
  -H "Authorization: Bearer $YUNXIN_API_KEY"

Capabilities depend on the pulled model and your hardware. The authoritative list for Yunxin is GET /v1/models?provider=ollama; locally, run ollama list or query http://localhost:11434/v1/models.

Local deployment — Run models on your own machine
Simple CLI — Easy model pull and run
Model library — Browse models at ollama.com/library
OpenAI-compatible API — Standard chat completions endpoint, no key required locally
Custom models — Import and run GGUF models

Quick start

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run a model
ollama run model-id

Calling Ollama directly (local)

Point the OpenAI SDK at your local Ollama server. No API key is required, so any non-empty placeholder works. Note the OpenAI-compatible routes live under the /v1 path.

from openai import OpenAI

client = OpenAI(
    api_key="ollama",
    base_url="http://localhost:11434/v1"
)

response = client.chat.completions.create(
    model="model-id",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "ollama",
  baseURL: "http://localhost:11434/v1",
});

const response = await client.chat.completions.create({
  model: "model-id",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(response.choices[0].message.content);

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "model-id",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Using Ollama through Yunxin

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["YUNXIN_API_KEY"],
    base_url="https://api.yuhuanstudio.com/v1"
)

response = client.chat.completions.create(
    model="ollama/model-id",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.YUNXIN_API_KEY,
  baseURL: "https://api.yuhuanstudio.com/v1",
});

const response = await client.chat.completions.create({
  model: "ollama/model-id",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(response.choices[0].message.content);

curl https://api.yuhuanstudio.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $YUNXIN_API_KEY" \
  -d '{
    "model": "ollama/model-id",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Available models

Ollama serves whichever models you have pulled. When accessed through Yunxin, query the Models API:

curl "https://api.yuhuanstudio.com/v1/models?provider=ollama" \
  -H "Authorization: Bearer $YUNXIN_API_KEY"

Overview

Key features

Quick start

Calling Ollama directly (local)

Using Ollama through Yunxin

Available models

Official resources

On this page

Ollama

Overview

Key features

Quick start

Calling Ollama directly (local)

Using Ollama through Yunxin

Available models

Official resources

On this page