Tool calling

Last updated:

On this page

staik supports OpenAI-compatible tool calling. Send tools and tool_choice in your request — the model responds with tool_calls in the exact same format as OpenAI. All chat models support tool calling; gemma4:31b and qwen3.6:35b-a3b have the best tool-following.

Define a tool

bash
curl https://api.staik.se/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-st-your-key" \
  -d '{
    "model": "gemma4:31b",
    "messages": [
      {"role": "user", "content": "What is the weather in Stockholm?"}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {"type": "string"},
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
          },
          "required": ["city"]
        }
      }
    }],
    "tool_choice": "auto"
  }'

The model responds with tool_calls instead of text when it wants to call a tool:

json
{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\": \"Stockholm\", \"unit\": \"celsius\"}"
        }
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

Full tool loop

python
import json
from openai import OpenAI

client = OpenAI(base_url="https://api.staik.se/v1", api_key="sk-st-your-key")

def get_weather(city: str, unit: str = "celsius") -> dict:
    # Call your real weather API here
    return {"city": city, "temp": 4, "unit": unit, "conditions": "cloudy"}

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "What is the weather in Stockholm?"}]

# 1. The model decides to call the tool
response = client.chat.completions.create(
    model="gemma4:31b", messages=messages, tools=tools,
)
msg = response.choices[0].message
messages.append(msg)

# 2. Execute each tool call and feed the result back
for tc in msg.tool_calls or []:
    args = json.loads(tc.function.arguments)
    result = get_weather(**args)
    messages.append({
        "role": "tool",
        "tool_call_id": tc.id,
        "content": json.dumps(result),
    })

# 3. The model formulates the final answer using the tool result
final = client.chat.completions.create(model="gemma4:31b", messages=messages)
print(final.choices[0].message.content)

Set tool_choice: "required" to force a tool call, or {"type": "function", "function": {"name": "..."}} for a specific tool. Streaming is supported — tool_calls arrive in the delta field using the same indexed format as OpenAI.

Web search (built-in tool)

Skip building search dependencies into every agent. Send "web_search": true (or add {"type": "web_search"} to tools) and staik runs the search for the model, looping the results back automatically:

  1. The model is given access to the web_search(query, max_results) tool
  2. If it needs current information, it calls the tool
  3. staik runs the search and feeds the results back
  4. The model answers — or searches again (up to an internal limit)

Tokens for all rounds are summed in usage, just like a regular call. Your own tools in the same request are not executed server-side — only web_search is run by staik.

bash
curl https://api.staik.se/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-st-your-key" \
  -d '{
    "model": "qwen3.6:35b-a3b",
    "messages": [
      {"role": "user", "content": "What happened recently in EU AI policy?"}
    ],
    "web_search": true
  }'

Clients that don't allow extra body fields can set the header X-Staik-Web-Search: 1 instead — equivalent to "web_search": true.

Just want raw search results (no model), e.g. for your own RAG pipeline:

bash
curl https://api.staik.se/v1/tools/web-search \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-st-your-key" \
  -d '{"query": "swedish LLM API", "max_results": 5}'

# => {"results": [{"title": "...", "url": "...", "content": "..."}, ...]}

Privacy

Search runs via a self-hosted search engine on staik's own hardware — no third-party search API. The query never leaves the infrastructure.

Structured JSON output

Need guaranteed parseable JSON? Send response_format and grammar-constrained decoding (guided decoding) is enabled server-side — the model cannot emit markdown fences or comments.

{"type": "json_object"} guarantees valid JSON but not which fields. To force an exact structure, use json_schema:

bash
curl https://api.staik.se/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-st-your-key" \
  -d '{
    "model": "gemma4:31b",
    "messages": [
      {"role": "user", "content": "Extract name and age: Anna is 30."}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "schema": {
          "type": "object",
          "properties": {"name": {"type": "string"}, "age": {"type": "integer"}},
          "required": ["name", "age"]
        }
      }
    }
  }'

Good to know about gemma4:31b

For strict structured output, gemma4:31b can rarely get stuck in a whitespace loop after valid JSON until max_tokens is spent (finish_reason: "length"). Send presence_penalty between 0.3 and 0.5 to break the loop. If you need maximum robustness, qwen3.6:35b-a3b is the more predictable choice. Also note: temperature: 0 is greedy per request but not bitwise reproducible across requests (continuous batching, MoE routing, prefix caching).