LiteLLM

Using LiteLLM proxy for unified LLM access with Upsonic

Overview

LiteLLM provides a unified OpenAI-compatible interface to access 100+ LLM providers including OpenAI, Anthropic, Azure, Google, AWS Bedrock, and more. Run it as a proxy server for centralized model management.

Model Class: OpenAIChatModel (OpenAI-compatible proxy)

Authentication

export LITELLM_BASE_URL="http://localhost:4000"
# Optional: API key if proxy requires authentication
export LITELLM_API_KEY="sk-..."

Examples

from upsonic import Agent, Task
from upsonic.models.openai import OpenAIChatModel

model = OpenAIChatModel(model_name="gpt-4o", provider="litellm")
agent = Agent(model=model)

task = Task("Hello, how are you?")
result = agent.do(task)
print(result)

Parameters

Parameter	Type	Description	Default	Source
max_tokens	int	Maximum tokens to generate	Model default	Base
temperature	float	Sampling temperature (0.0-2.0)	1.0	Base
top_p	float	Nucleus sampling	1.0	Base
seed	int	Random seed	None	Base
stop_sequences	list[str]	Stop sequences	None	Base
presence_penalty	float	Token presence penalty	0.0	Base
frequency_penalty	float	Token frequency penalty	0.0	Base
parallel_tool_calls	bool	Allow parallel tools	True	Base
timeout	float	Request timeout (seconds)	600	Base