Ollama

Using Ollama for local model deployment with Upsonic

Overview

Ollama allows you to run large language models locally on your machine. Perfect for development, testing, and privacy-sensitive applications.

Model Class: OpenAIChatModel (OpenAI-compatible API)

Authentication

export OLLAMA_BASE_URL="http://localhost:11434/v1/"  # Optional, this is the default

Examples

from upsonic import Agent, Task
from upsonic.models.openai import OpenAIChatModel

model = OpenAIChatModel(model_name="llama3.2", provider="ollama")
agent = Agent(model=model)

task = Task("Hello, how are you?")
result = agent.do(task)
print(result)

Parameters

Parameter	Type	Description	Default	Source
max_tokens	int	Maximum tokens to generate	Model default	Base
temperature	float	Sampling temperature	0.8	Base
top_p	float	Nucleus sampling	0.9	Base
seed	int	Random seed	None	Base
stop_sequences	list[str]	Stop sequences	None	Base