sie

Open source alternative to Baseten and Fireworks AI

3.4K stars
Apache-2.0
Active this week
Last commit today

Last verified Oct 21, 2026

Serve all the open-model inference your AI agents need from a self-hosted Kubernetes cluster, replacing per-token APIs with compute-cost-only pricing.

Visit website GitHub3.4K Deploy on Hostinger

Read the full breakdownWho it is for, how it works, trade-offs, the quick start and the FAQ

who it's for

Who sie is for#

AI engineers building production RAG pipelines

SIE serves the full retrieval stack from one cluster: embedding documents with bge-m3 or SPLADE, reranking with ColBERT or qwen3-reranker, and parsing PDFs with GLM-OCR or PaddleOCR-VL. Teams eliminate separate API billing for each step and keep all data inside their own network.

Skip if:

You are prototyping with low query volume. A single managed API call is simpler when you are not yet at the scale where per-token fees are meaningful and Kubernetes overhead is justified.

Compliance-constrained teams that cannot send data to third-party APIs

SIE runs fully within your VPC or on-premises. Prompts, documents, and embeddings never leave your network. Air-gapped installs pull models from mirrored snapshots, making it usable in classified or regulated environments.

Skip if:

Your compliance requirements permit sending data to SOC2 Type 2-certified managed services. In that case, Fireworks AI or Baseten may offer faster time-to-production with less infrastructure overhead.

Teams migrating agent code from OpenAI-compatible APIs

SIE's API matches the OpenAI spec at /v1/embeddings, /v1/chat/completions, /v1/completions, and /v1/responses. A one-line base URL change migrates most existing agent code. Python and TypeScript SDKs are available alongside the REST API.

Skip if:

Your agents depend on frontier model capabilities (GPT-5, Claude, Gemini). SIE serves open models only; it is not a proxy for frontier-lab APIs and does not match their performance on general reasoning tasks.

MLOps engineers operating multi-model inference clusters

SIE's POOL-THEN-BATCH architecture routes mixed model requests through a cluster-wide queue, improving GPU utilization from roughly 51% (worker-local) to 89%. KEDA autoscaling handles scale-to-zero and burst traffic automatically.

Skip if:

Your inference workload is a single model type at predictable volume. A dedicated model server (vLLM, TEI) is simpler to operate without SIE's orchestration layer.

the problem

The problem it solves#

Building production AI agents typically means paying multiple inference APIs simultaneously: one for embeddings, another for reranking, a third for chat completion, and more for OCR or structured extraction. Each call goes to a different third-party server, so prompts and documents leave your infrastructure with every request. At scale, per-token fees from commercial providers add up to several times the actual compute cost.

The harder problem is orchestration. Each model type needs a different runtime, so teams end up managing multiple model servers with no shared batching or resource pooling. Worker-local queue architectures run GPU resources unevenly, keeping utilization well below 70%. Teams either accept that inefficiency or spend engineering effort building custom queuing on top of existing model servers that were not designed to share GPU capacity across model families.

how sie solves it

How it solves it#

Cluster-wide queue with 89% GPU utilization

SIE routes all inference work through a shared pool queue rather than per-worker queues. Mixed request sizes batch cleanly against the full GPU pool, reaching 89% GPU efficiency versus roughly 51% in worker-local architectures. This means fewer GPUs serve the same throughput at production volume.

OpenAI-compatible API for drop-in migration

SIE publishes /v1/embeddings, /v1/chat/completions, /v1/completions, and /v1/responses. Migrating agent code to SIE requires only a base_url change; no SDK rewrites needed. Python and TypeScript SDKs are available alongside the REST API.

Five agent task types in one server

One cluster serves search (embed, match, rerank), document-to-markdown conversion, structured JSON extraction, content safety classification, and the agent loop LLM. Key models: bge-m3, splade-v3, colbertv2, qwen3-reranker, glm-ocr, gliner2, granite-guardian-2b, and qwen3.6-27b.

Helm charts and Terraform for production Kubernetes

SIE ships Helm charts with KEDA autoscaling (scale to zero), a load-balancing gateway, and Grafana dashboards. Public Terraform modules cover AWS EKS, Google GKE, Azure AKS, and Alibaba Cloud ACK. Air-gapped installs run from mirrored model snapshots.

100+ models with on-demand loading and LRU eviction

The model catalog covers embeddings (bge-m3, Stella, SPLADE, ColBERT), rerankers, OCR engines (GLM-OCR, PaddleOCR-VL, MinerU, Docling), entity extractors (GLiNER), safety classifiers (Granite Guardian), and LLMs (Qwen3 family). Models load on first call and evict under memory pressure.

strengths · trade-offs

Strengths and trade-offs#

Strengths

Apache 2.0 license with no usage restrictionsSIE is Apache 2.0 licensed across the full stack, including Helm charts and Terraform modules. Unlike managed inference APIs that bill per token, you run SIE on your own GPUs with no per-request fees and no restrictions on commercial use or modification.
Data never leaves your infrastructureAll inference runs inside your cloud or on-premises network. Prompts, documents, and model outputs stay within your network boundary. This is essential for teams subject to GDPR, HIPAA, or internal data-classification policies that prohibit sending text to third-party APIs.
Integrates with nine agent frameworks and vector storesSIE integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB. The OpenAI Agents SDK is supported via a one-line base URL change, with LangGraph and CrewAI integrations noted as upcoming.
MTEB-benchmarked model catalogEmbedding and retrieval models in the catalog are benchmarked on MTEB tasks, so you can select models by documented task performance. The catalog cites bge-m3 as 2.7x faster than Cohere rerank-3.5 on MTEB AskUbuntu and Qwen3.6-27B at 96% on the AA Intelligence Index versus GPT-5.1.

Trade-offs

-Kubernetes expertise required for production deploymentRunning SIE at production scale requires operating a Kubernetes cluster: managing GPU node pools, Helm releases, KEDA autoscaling, and Grafana dashboards. Teams without existing Kubernetes infrastructure face significant setup time before their first production inference call.
-GPU hardware required for high-performance inferenceMost catalog models need NVIDIA GPUs (L4 or H100 class). The CPU image runs lightweight models for development, but embedding quality and LLM generation performance degrade significantly. Cloud GPU instances add ongoing cost that only offsets per-token API savings at sustained inference volume.
-Anonymous telemetry enabled by defaultSIE collects anonymous usage data (version, OS, architecture, GPU type) by default. Disabling it requires setting SIE_TELEMETRY_DISABLED=1 or DO_NOT_TRACK=1. No IP addresses or request data are collected, but the opt-out requirement may conflict with strict data-governance policies.

versus alternatives

sie vs alternatives#

SIE vs Fireworks AI

Fireworks AI is a managed inference API optimized for fast open-model serving, primarily LLMs and embeddings. SIE is a self-hosted inference cluster. The decision between them comes down to where your data must live and whether your team can operate Kubernetes.

Feature	SIE	Fireworks AI
License	Apache 2.0	Proprietary
Deployment	Self-hosted (Kubernetes, Docker)	Managed API only
Data residency	Full (stays in your cloud)	Requests sent to Fireworks servers
Model types	LLMs, embeddings, rerankers, OCR, safety, structured output	LLMs and embeddings
Pricing	Compute cost only (no per-token fees)	Per-token billing
Air-gapped support	Yes	No

SIE is the better choice when your data cannot leave your infrastructure, when you are running sustained inference volume where per-token fees are significant, or when you need model types beyond LLMs and embeddings (OCR, safety classifiers, entity extractors). Fireworks AI is a better fit when you need fast time-to-production, lack Kubernetes expertise, or have bursty or low-volume workloads where the overhead of running a Kubernetes cluster outweighs per-token costs.

SIE vs Baseten

Baseten is a model serving platform that handles deployment, autoscaling, and GPU management for custom and open models. SIE covers similar ground but runs entirely in your own cloud account, giving you full data residency.

Feature	SIE	Baseten
License	Apache 2.0	Proprietary
Deployment	Self-hosted	Managed
Data residency	Full (your cloud)	Requests process on Baseten servers
Model catalog	100+ pre-configured open models	Custom models plus curated set
Terraform support	Yes (AWS, GCP, Azure, Alibaba Cloud)	Not applicable
Pricing	Compute cost only	Per-compute-second billing

Baseten is well-suited for teams that want to serve custom fine-tuned models without managing infrastructure themselves. SIE is the better fit when the full catalog of agent models (embedding, reranking, OCR, safety) needs to run in a single self-hosted cluster with no data leaving your account. Teams with strict compliance requirements (GDPR, HIPAA, air-gapped environments) typically favor SIE's self-hosted deployment model.

install · quick start

Quick start#

bash

Self-hosting uses Docker for local development and Helm for production Kubernetes clusters.
```bash
pip install "sie-server[local]" && sie-server serve

docker run --gpus all -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  ghcr.io/superlinked/sie-server:latest-cuda12-default

pip install sie-sdk

npm install @superlinked/sie-sdk
```

tech stack · detected from GitHub

What it's built on#

Languages: PythonRustTypeScript

frequently asked

FAQ#

Is SIE free to use?

SIE is Apache 2.0 licensed and free to self-host. The server, Helm charts, and Terraform modules are all open source with no usage fees. A managed cloud tier is listed as upcoming on the website but is not yet available; the current free path is self-hosting on your own infrastructure.

What models does SIE support?

SIE's catalog covers over 100 models across five task types: embeddings (bge-m3, SPLADE, Stella, ColBERT), rerankers (qwen3-reranker, jina-reranker), OCR (GLM-OCR, PaddleOCR-VL, MinerU, Docling), entity extractors (GLiNER), safety classifiers (Granite Guardian), and LLMs (Qwen3 family). You can also pass any Hugging Face model ID directly to the API.

How does SIE compare to Fireworks AI or Baseten?

Fireworks AI and Baseten are managed inference APIs: you send requests to their servers, pay per token or per compute-second, and they handle infrastructure. SIE runs in your own cloud account, eliminates per-token fees, and keeps all data inside your network. The tradeoff is that SIE requires Kubernetes expertise to operate at production scale, whereas managed APIs work immediately with no infrastructure setup.

Can I run SIE without a GPU?

Yes. SIE ships a CPU image that runs lighter models for development and testing. Production embedding and generation quality are lower on CPU-only hardware, and inference latency is higher. Most teams use the CPU image locally and NVIDIA GPU instances (L4 or H100 class) in production.

Does SIE work with my existing agent framework?

SIE is OpenAI-API-compatible and integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB. A one-line base URL change in the OpenAI SDK migrates most existing agent integrations without any other code changes.

also worth a look

Similar open-source tools#

Vllm

Alternative toTogether AI

High-throughput LLM inference server with OpenAI-compatible API

94,441 GitHub stars
Licence Apache-2.0

Tokenspeed

Alternative toFireworks AI

MIT-licensed LLM inference engine built for agentic coding workloads.

2,715 GitHub stars
Licence MIT

AirLLM

Alternative toHugging Face Inference Endpoints

Run huge LLMs on low-end GPUs with minimal VRAM

39,915 GitHub stars
Licence Apache-2.0

LMCache

Alternative toAmazon SageMaker

Accelerate AI applications with caching technology

12,129 GitHub stars
Licence Apache-2.0

Omnara

Featured

Alternative toClaude Managed Agents

Open-source agent deployment API. Self-host or use Omnara Cloud.

2,957 GitHub stars
Licence Apache-2.0

Hermes Agent

Featured

Alternative toGrok Bot

Self-hosted AI agent: persistent memory, multi-channel chat, model choice.

255,514 GitHub stars
Licence MIT

by category

Open source tools in these categories#

404

Run sie on your own server

Self-host sie on a Hostinger VPS.

Deploy on Hostinger

Featured

Inbox Zero

Alternative toSuperhuman

AI email assistant that drafts replies and clears your inbox

12,712 GitHub stars
Licence AGPL-3.0
#Email

rakazo

Alternative toGrok Bot

AI teammates you own: your keys, your model, your machine.

4,904 GitHub stars
Licence Apache-2.0
#AI Agents

Omnara

Alternative toClaude Managed Agents

Open-source agent deployment API. Self-host or use Omnara Cloud.

2,957 GitHub stars
Licence Apache-2.0
#AI Agents

Feature your project

The projects we add and every new guide. Free, unsubscribe any time.

Find open source alternatives to paid software

SIE vs Fireworks AI

Feature	SIE	Fireworks AI
License	Apache 2.0	Proprietary
Deployment	Self-hosted (Kubernetes, Docker)	Managed API only
Data residency	Full (stays in your cloud)	Requests sent to Fireworks servers
Model types	LLMs, embeddings, rerankers, OCR, safety, structured output	LLMs and embeddings
Pricing	Compute cost only (no per-token fees)	Per-token billing
Air-gapped support	Yes	No

SIE vs Baseten

Feature	SIE	Baseten
License	Apache 2.0	Proprietary
Deployment	Self-hosted	Managed
Data residency	Full (your cloud)	Requests process on Baseten servers
Model catalog	100+ pre-configured open models	Custom models plus curated set
Terraform support	Yes (AWS, GCP, Azure, Alibaba Cloud)	Not applicable
Pricing	Compute cost only	Per-compute-second billing

Who sie is for#

AI engineers building production RAG pipelines

Skip if:

Compliance-constrained teams that cannot send data to third-party APIs

Skip if:

Teams migrating agent code from OpenAI-compatible APIs

Skip if:

MLOps engineers operating multi-model inference clusters

Skip if:

The problem it solves#

How it solves it#

Cluster-wide queue with 89% GPU utilization

OpenAI-compatible API for drop-in migration

Five agent task types in one server

Helm charts and Terraform for production Kubernetes

100+ models with on-demand loading and LRU eviction

Strengths and trade-offs#

Strengths

Trade-offs

sie vs alternatives#

SIE vs Fireworks AI

SIE vs Baseten

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

AI & Machine Learning

Cloud & Hosting

Developer Tools

LLMOps & AI Tooling

Find open source alternatives to paid software

Who sie is for#

AI engineers building production RAG pipelines

Skip if:

Compliance-constrained teams that cannot send data to third-party APIs

Skip if:

Teams migrating agent code from OpenAI-compatible APIs

Skip if:

MLOps engineers operating multi-model inference clusters

Skip if:

The problem it solves#

How it solves it#

Cluster-wide queue with 89% GPU utilization

OpenAI-compatible API for drop-in migration

Five agent task types in one server

Helm charts and Terraform for production Kubernetes

100+ models with on-demand loading and LRU eviction

Strengths and trade-offs#

Strengths

Trade-offs

sie vs alternatives#

SIE vs Fireworks AI

SIE vs Baseten

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

AI & Machine Learning

Cloud & Hosting

Developer Tools

LLMOps & AI Tooling

Find open source alternatives to paid software