Soup

Open source alternative to Together AI, Hugging Face AutoTrain and Google Cloud Vertex AI

9.5K stars
Apache-2.0
Active this week
Last commit 1 day ago

Last verified Oct 20, 2026

Build fine-tuned LLMs from one YAML, running Llama-3.1-8B on a 4 GB laptop GPU via layer streaming. Apache-2.0 licensed, 23 training methods.

Visit website GitHub9.5K

Read the full breakdownWho it is for, how it works, trade-offs, the quick start and the FAQ

who it's for

Who Soup is for#

ML engineers fine-tuning on local consumer hardware

Soup's layer streaming lets a single 4 GB GPU run fine-tuning jobs that previously required renting cloud compute. The YAML-first workflow and auto-detection handle most config decisions, so engineers spend time on data and evaluation rather than tuning training parameters.

Skip if:

Your team has dedicated high-VRAM GPUs (24 GB+) where a resident run is straightforward, or your organization already has managed training infrastructure. Layer streaming's throughput penalty is not worth taking on capable hardware.

Researchers building preference-trained or RLHF models

DPO, ORPO, SimPO, KTO, GRPO, and PPO all run through the same YAML interface. The `soup advise` command ranks training methods against your data before you commit GPU hours. `soup data doctor` runs 8 chat-template checks that catch the EOS bug and other silent data failures before training starts.

Skip if:

Your preference training requires online DPO against a remote judge model that cannot run locally. GRPO and PPO with online reward models have specific infrastructure requirements not covered by the consumer-GPU path.

Teams migrating from LLaMA-Factory, Axolotl, or Unsloth

`soup migrate` converts existing configs in one command and preserves training hyperparameters. Teams can try Soup's `soup ship` release gate and semantic data dedup without rewriting their entire pipeline first.

Skip if:

Your existing config uses advanced features like custom data pipelines, non-standard PEFT variants, or multi-node setups that may not convert cleanly. Verify the output config against your expected training behavior before committing to the migration.

Organizations that need compliance-ready ML pipelines

`soup init --template hipaa` (or soc2, eu-ai-act, sr-11-7) starts from a regulation-shaped config. `soup bom` and `soup attest` emit signed ML-BOMs and SLSA-3 provenance. `soup ci init` blocks merges on DON'T SHIP verdicts. For teams with audit requirements, this is a self-hosted path to documented, reproducible, signed training runs.

Skip if:

Your compliance requirements mandate a certified managed service with SLA guarantees. Self-hosted tooling satisfies technical controls but does not substitute for vendor certification in all frameworks.

the problem

The problem it solves#

LLM fine-tuning demands either significant cloud spend or deep infrastructure knowledge. Managed training services from providers like Together AI, Hugging Face AutoTrain, and Google Cloud Vertex AI charge per GPU-hour, require uploading your data to their servers, and lock your workflow into proprietary tooling. Rolling your own training stack means wiring together PyTorch, PEFT, TRL, quantization libraries, and experiment tracking by hand, with no pre-flight checks to catch silent failures before they waste GPU hours.

The deeper problem is hardware: most developers and researchers work on consumer GPUs with 4 to 8 GB of VRAM, which is not enough to load a 7 to 8 billion parameter model resident in memory, let alone train one. That ceiling has historically meant renting cloud GPUs for every experiment, even quick iteration runs that do not justify the cost.

how Soup solves it

How it solves it#

Layer Streaming (BETA)

Streams the frozen base model from CPU RAM or NVMe one decoder layer at a time, quantizing to NF4 on the fly. Measured result: Llama-3.1-8B trains at 119.6 tok/s in 3.32 GB peak VRAM on a 4 GB RTX 3050 Laptop. DPO, ORPO, SimPO, and KTO stream too; the DPO reference model reuses the same streamed base at 0.914x the supervised peak, adding no extra weights. Currently BETA and limited to transformers, text, and plain LoRA.

23 Training Methods, 142 Ready Recipes

Supports SFT, DPO, ORPO, SimPO, KTO, GRPO, PPO, IPO, BCO, distillation, classification, vision, audio, and more across 23 methods. 142 pre-built recipes cover major model families so most tasks start from a tested baseline rather than a blank config. `soup recipes list` shows the full catalog.

Automatic Config Generation

`soup init` runs an interactive wizard or starts from a named template (chat, code, tool-calling, medical, reasoning, vision, and 15 more). Soup auto-detects batch size, target modules, optimizer, and scheduler from the task and hardware, eliminating the most common causes of misconfigured training runs.

`soup ship`: Release Gate for Weights

Runs the fine-tuned model against seven offline suites (MCQ, arithmetic, tool-calling, JSON validity, safety/refusal) and returns a binary SHIP or DON'T SHIP verdict with committable evidence. `--noise-floor N` re-runs the base model N times to measure GPU non-determinism before calling a delta significant. Exit codes: 0 = SHIP, 2 = DON'T SHIP, 1 = runtime error.

Migration from LLaMA-Factory, Axolotl, and Unsloth

`soup migrate --from llamafactory config.yaml` converts an existing config to soup.yaml format automatically. Supports LLaMA-Factory, Axolotl, and Unsloth notebook conversion. No manual rewriting; the converter handles field mapping and adds Soup's auto-detected parameters for what it cannot infer.

MCP Server for Coding Agents

`soup mcp serve` exposes Soup to any MCP client (Claude Code, Cursor, Cline, Continue) over stdio. 14 read-only tools return JSON; 2 plan-only mutating tools allow training without a terminal. No network listener required.

strengths · trade-offs

Strengths and trade-offs#

Strengths

Trains 8B Models on a 4 GB Consumer GPUThe layer-streaming BETA is the only open source training path that can fit Llama-3.1-8B on a card with 4 GB of VRAM, measured on a real RTX 3050 Laptop and independently verified on a borrowed 8x H100. For developers without a cloud budget or a high-VRAM GPU, this is the enabling capability. Comparable cloud training runs on Together AI or Vertex AI cost money per GPU-hour and send data off-premise.
Honest, Disclosed BenchmarksEvery measurement on the Soup website and README identifies the hardware, the version, and what changed since. The v0.73.0 release notes document a silent gradient bug at 32B+ NF4 (fixed), a prior adapter-key bug that made adapters load as a no-op (fixed in v0.72.1), and three withdrawn readings. This level of disclosure is uncommon in the space and makes performance numbers trustworthy.
Apache-2.0 License with No Usage RestrictionsApache-2.0 allows commercial use, modification, and distribution without source-publishing obligations. There is no enterprise tier required for production use, no API key tied to training runs, and no telemetry. The source is on GitHub under the same license used for deployment.
Integrates Across the ML StackFirst-class support for Unsloth (2-5x faster training), Apple MLX (M1 to M4 native), DeepSpeed (ZeRO 2/3/++), FlashAttention v2/v3, vLLM, SGLang, Ollama, llama.cpp, HuggingFace Hub push, and W&B/MLflow/SwanLab experiment tracking. Models train on Soup and deploy with the same tools the team already uses.

Trade-offs

-Layer Streaming Is Still BETALayer streaming is limited to transformers, text, and plain LoRA. The reference model costs no extra memory but reads the layer stack 1.52x as often per step, slowing training versus a resident run. The 32B+ NF4 backward-pass bug (fixed in v0.73.0) and the v0.72.0 adapter-key silent no-op bug (fixed in v0.72.1) show that edge cases in this path are still being found. No tok/s is claimed for preference losses on 4 GB hardware because none was measured.
-Python 3.10 to 3.12 OnlyPython 3.13 is not supported. On 3.13+, pip resolves untested PyTorch wheels that crash in the native extension before Soup starts. If your environment is pinned to a newer Python version, you need a virtual environment or container on a supported version before installing.
-Self-Hosting Requires a GPU for TrainingCPU training is experimental and very slow per the README. Apple Silicon (MPS) works but requires the `[mlx]` extra. Production-quality fine-tuning requires a CUDA GPU. If you have no GPU access and cannot use cloud credits, Soup's value is limited to data tooling and model serving rather than training.

versus alternatives

Soup vs alternatives#

Soup vs Together AI

Both Soup and Together AI offer fine-tuning for LLMs including Llama and Qwen family models, but they operate on opposite sides of the build-vs-buy line. Together AI is a managed cloud API; Soup is a self-hosted CLI.

Feature	Soup	Together AI
License	Apache-2.0	Proprietary
Self-hosting	Yes, required	No
Runs on 4 GB consumer GPU	Yes (layer streaming BETA)	No
Data leaves your machine	No	Yes
Pricing	Free (your hardware)	Per-GPU-hour
Training methods	23 methods, 142 recipes	Managed selection

Soup is the stronger choice when your training data is sensitive, your budget is limited, or you need to iterate quickly on a consumer GPU. Together AI is worth considering when you need zero infrastructure management, access to high-VRAM clusters for models above 70B, or managed multi-node training without configuring DeepSpeed yourself.

Soup vs Hugging Face AutoTrain

Hugging Face AutoTrain is a managed fine-tuning service on the HuggingFace platform. Like Soup, it supports LoRA fine-tuning for popular model families. Unlike Soup, it runs on Hugging Face's cloud infrastructure, requires uploading your dataset to their servers, and charges per training run.

Feature	Soup	HF AutoTrain
License	Apache-2.0	Proprietary
Self-hosting	Yes	No (cloud-only)
Dataset privacy	Stays local	Uploaded to HF
Config format	soup.yaml	Web UI or YAML
Release gating	soup ship verdict	Manual
Migration support	From LLaMA-Factory, Axolotl, Unsloth	None

Soup's soup ship release gate, semantic data dedup, and compliance templates (HIPAA, SOC2, EU-AI-Act) have no equivalent in AutoTrain. Teams that need auditable, reproducible training pipelines with signed provenance will find Soup more capable. AutoTrain is more accessible for one-off fine-tuning without any local infrastructure.

Soup vs Google Cloud Vertex AI

Vertex AI is a full ML platform covering model training, serving, and MLOps at enterprise scale. Compared to Vertex AI, Soup is narrower in scope (fine-tuning and post-training, not the full ML lifecycle) but requires no GCP account, no cloud spend, and no data egress. Vertex AI is the better choice for organizations already committed to GCP that need managed training at scale, AutoML pipelines, or integrated model monitoring. Soup is the better fit for teams that want self-hosted training on their own hardware with no vendor dependency.

install · quick start

Quick start#

bash

Install Soup via pip with the training extra, then run soup init to create a config and soup train to begin training.
```bash
pip install "soup-cli[train]"
soup init
soup train
```

tech stack · detected from GitHub

What it's built on#

Languages: Python
Frameworks: FastAPIPyTorch

frequently asked

FAQ#

Can Soup really fine-tune an 8B model on a 4 GB GPU?

Yes, with layer streaming enabled (stream_layers: true in soup.yaml), which is currently BETA. The approach streams the frozen base model from CPU RAM or NVMe one decoder layer at a time instead of loading it fully into VRAM. Measured result: Llama-3.1-8B-Instruct at 119.6 tok/s in 3.32 GB peak on an RTX 3050 Laptop 4 GB. The measurement was taken on v0.72.2; a v0.73.0 correctness repair cost 4.8% throughput at 32B and has not been re-run on a 4 GB card. You can verify it yourself on a free Colab T4 using the published notebook in the repo.

Is Soup free to use commercially?

Yes. Soup is Apache-2.0 licensed, which permits commercial use, modification, and distribution without source-publishing obligations. There is no paid tier required for production use, no per-run API fee, and no telemetry. Training runs stay entirely on your own hardware.

How does Soup compare to Together AI or Hugging Face AutoTrain?

Together AI and Hugging Face AutoTrain are managed cloud services: you upload your data, they run training on their infrastructure, and you pay per GPU-hour. Soup is a self-hosted CLI: you run it on your own GPU, your data never leaves your machine, and the per-run cost is electricity. The tradeoff is that you manage the server and GPU, and some features (multi-GPU at scale, Apple Silicon) require additional setup. For teams with data-privacy requirements or tight cloud budgets, Soup is the stronger fit.

What Python versions does Soup support?

Python 3.10, 3.11, and 3.12. Python 3.13 is not supported because the PyTorch stack has not been validated on that version; pip resolves untested wheels that crash in the native extension before Soup starts. Use a virtual environment pinned to a supported version if your system Python is newer.

Can I migrate from LLaMA-Factory or Axolotl to Soup?

Yes. soup migrate --from llamafactory config.yaml or soup migrate --from axolotl config.yml converts your existing config to soup.yaml format in one command. Soup auto-detects parameters it can infer and fills them in. Review the output config before your first training run to catch anything the converter could not map, especially if your original config used non-standard PEFT variants or custom data pipelines.

also worth a look

Similar open-source tools#

Unsloth

Alternative toTogether AI

Train LLMs locally without code using a browser-based interface

78,315 GitHub stars
Licence Apache-2.0

Vllm

Alternative toTogether AI

High-throughput LLM inference server with OpenAI-compatible API

94,441 GitHub stars
Licence Apache-2.0

LocalAI

Alternative toOpenAI API

Self-hosted AI runtime: any model, any modality, any hardware.

49,798 GitHub stars
Licence MIT

Tokenspeed

Alternative toFireworks AI

MIT-licensed LLM inference engine built for agentic coding workloads.

2,715 GitHub stars
Licence MIT

accelerate

Alternative toLightning AI

Distribute PyTorch training to multi-GPU and TPU with four lines

9,922 GitHub stars
Licence Apache-2.0

marin

Alternative toDatabricks

Open lab for training foundation models together

3,987 GitHub stars
Licence Apache-2.0

by category

Open source tools in these categories#

404

4,904 GitHub stars
Licence Apache-2.0
#AI Agents

Inbox Zero

Alternative toSuperhuman

AI email assistant that drafts replies and clears your inbox

12,712 GitHub stars
Licence AGPL-3.0
#Email

Omnara

Alternative toClaude Managed Agents

Open-source agent deployment API. Self-host or use Omnara Cloud.

2,957 GitHub stars
Licence Apache-2.0
#AI Agents

Feature your project

The projects we add and every new guide. Free, unsubscribe any time.

Find open source alternatives to paid software

Soup vs Together AI

Feature	Soup	Together AI
License	Apache-2.0	Proprietary
Self-hosting	Yes, required	No
Runs on 4 GB consumer GPU	Yes (layer streaming BETA)	No
Data leaves your machine	No	Yes
Pricing	Free (your hardware)	Per-GPU-hour
Training methods	23 methods, 142 recipes	Managed selection

Soup vs Hugging Face AutoTrain

Feature	Soup	HF AutoTrain
License	Apache-2.0	Proprietary
Self-hosting	Yes	No (cloud-only)
Dataset privacy	Stays local	Uploaded to HF
Config format	soup.yaml	Web UI or YAML
Release gating	soup ship verdict	Manual
Migration support	From LLaMA-Factory, Axolotl, Unsloth	None

Who Soup is for#

ML engineers fine-tuning on local consumer hardware

Skip if:

Researchers building preference-trained or RLHF models

Skip if:

Teams migrating from LLaMA-Factory, Axolotl, or Unsloth

Skip if:

Organizations that need compliance-ready ML pipelines

Skip if:

The problem it solves#

How it solves it#

Layer Streaming (BETA)

23 Training Methods, 142 Ready Recipes

Automatic Config Generation

`soup ship`: Release Gate for Weights

Migration from LLaMA-Factory, Axolotl, and Unsloth

MCP Server for Coding Agents

Strengths and trade-offs#

Strengths

Trade-offs

Soup vs alternatives#

Soup vs Together AI

Soup vs Hugging Face AutoTrain

Soup vs Google Cloud Vertex AI

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

AI & Machine Learning

Developer Tools

LLMOps & AI Tooling

Find open source alternatives to paid software

Who Soup is for#

ML engineers fine-tuning on local consumer hardware

Skip if:

Researchers building preference-trained or RLHF models

Skip if:

Teams migrating from LLaMA-Factory, Axolotl, or Unsloth

Skip if:

Organizations that need compliance-ready ML pipelines

Skip if:

The problem it solves#

How it solves it#

Layer Streaming (BETA)

23 Training Methods, 142 Ready Recipes

Automatic Config Generation

`soup ship`: Release Gate for Weights

Migration from LLaMA-Factory, Axolotl, and Unsloth

MCP Server for Coding Agents

Strengths and trade-offs#

Strengths

Trade-offs

Soup vs alternatives#

Soup vs Together AI

Soup vs Hugging Face AutoTrain

Soup vs Google Cloud Vertex AI

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

AI & Machine Learning

Developer Tools

LLMOps & AI Tooling

Find open source alternatives to paid software