Last verified
Analyze your hardware and rank LLM models by memory fit, speed, and quality. MIT-licensed Rust CLI with interactive TUI for Ollama, llama.cpp, and MLX.
llmfit ranks every model in its catalog against your workstation's VRAM and RAM before you download anything. The ranked table covers fit, speed, and context length in one view, replacing the trial-and-error cycle of loading and rejecting models one at a time.
You are running inference on a cloud GPU cluster with fixed hardware. llmfit adds no value when you are not selecting a model for hardware you own.
Dynamic quantization selection lets you see how different quant levels affect memory fit and estimated speed for the same base model on your hardware. The `llmfit info` command shows the estimate basis for any model-quant combination before you commit to a download.
You already know the exact model and quantization you need and just want to run it. llmfit is most useful in the selection phase, not the serving phase.
The built-in bench flow measures real tok/s and time-to-first-token, saves results locally, and lets you submit them back to the project as a pull request from the TUI. Each merged submission improves estimate accuracy for anyone running identical hardware.
Your organization prohibits contributing usage data or submitting pull requests to public repositories. Submission is fully opt-in; results are always saved locally first.
Running local LLMs on personal or workstation hardware is a trial-and-error process. A model that fits in VRAM on paper may swap to system RAM at runtime, dropping inference speed from tens of tokens per second to single digits. MoE architectures like DeepSeek-V3 complicate estimates further: naive memory calculations use total parameter count rather than the active subset, overestimating RAM requirements by a wide margin.
Most developers resort to reading community posts, checking model cards, or downloading and running models one at a time. None of those approaches account for the interaction between your specific CPU, GPU, VRAM, and system RAM. The result is wasted time, failed loads, and wasted storage before any actual inference work begins.
llmfit reads your RAM, CPU, GPU, and VRAM on launch and scores every model in its catalog across four dimensions: memory fit, estimated speed, quality, and context length. The score reflects your actual hardware configuration, not a generic recommendation.
Token-per-second estimates come from a memory-bandwidth formula grounded in runtime sampling and real community benchmarks. The `llmfit info` command shows the exact inputs for each estimate and the commands to verify them on your machine.
The default interactive TUI shows your detected hardware at the top and every model ranked in real time. CLI mode supports scripts and automation pipelines: `llmfit fit` prints a ranked table, and `llmfit recommend --json` returns top picks as JSON for agent or script consumption.
llmfit understands Mixture-of-Experts architectures and estimates memory from the active parameter subset rather than total parameter count. This produces accurate fit results for models like Mixtral and DeepSeek-V3, which standard memory calculators overestimate significantly.
A built-in benchmarking flow lets you measure real tok/s and time-to-first-token against your running local provider, then submit results back to the project as a pull request from the TUI. No GitHub CLI or third-party account required; results are saved locally before any submission.
Supports Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio as inference backends. The `llmfit doctor` command generates a hardware detection report for debugging provider integration issues.
NVIDIA AI Enterprise is an enterprise platform for deploying, managing, and scaling AI workloads on NVIDIA GPU infrastructure. It targets organizations running models in production at scale, with support for multi-node clusters, enterprise security, and managed inference microservices.
| Feature | llmfit | NVIDIA AI Enterprise |
|---|---|---|
| License | MIT | Proprietary |
| Deployment | Local binary, no server | Managed on NVIDIA hardware |
| Target scale | Single machine or workstation | Multi-node GPU clusters |
| Cost | Free | Per-GPU enterprise licensing |
| Hardware scope | Any hardware you own | NVIDIA GPUs only |
llmfit is the better choice when your goal is to understand what runs on hardware you already own, select the right model before downloading it, or build a local inference stack without a GPU fleet. NVIDIA AI Enterprise is the appropriate choice for production deployments on NVIDIA GPU infrastructure, managed model serving at scale, and enterprise compliance requirements that a single-binary CLI cannot address.
OctoAI is a managed cloud inference platform that runs optimized model endpoints on behalf of the caller. You call an API; OctoAI handles GPU allocation, model loading, and scaling. It is designed for teams that want fast inference without managing hardware.
| Feature | llmfit | OctoAI |
|---|---|---|
| License | MIT | Proprietary |
| Deployment | Local, no server required | Managed cloud API |
| Hardware | Your own machine | None (cloud) |
| Cost | Free | Per-token API pricing |
| Offline capable | Yes | No |
llmfit makes sense when you run models locally, want to avoid per-token API costs, or need offline inference. OctoAI is better when you need production-grade latency guarantees, want to avoid managing hardware entirely, or your workload exceeds what a single machine can serve. The two tools address different parts of the LLM stack: llmfit helps you select and fit models for your own hardware, while OctoAI removes the hardware question by hosting inference in the cloud.
Install llmfit via Homebrew on macOS/Linux, pip via uv, or Docker for a containerized run.
```bash
brew install llmfit
uv tool install -U llmfit
docker run ghcr.io/alexsjones/llmfit
```llmfit does not download models. It detects your hardware, scores models from its catalog, and tells you which ones fit and how fast they will run. Downloading and serving happen through your local runtime provider, such as Ollama or llama.cpp. The bench command measures performance after you have already loaded a model through your provider.
Yes. llmfit is MIT licensed and free to install and run with no account, API key, or paid tier. It contacts external services only when you explicitly trigger model downloads, provider queries, or the community leaderboard.
Speed estimates use a memory-bandwidth model grounded in runtime sampling and community benchmarks. They are accurate enough for useful model comparisons on the same hardware, but not a substitute for measured results. The llmfit bench command replaces estimates with real tok/s numbers from your running provider on your actual hardware.
Yes. MLX is one of the supported local runtime providers. llmfit handles Apple Silicon GPU detection and uses MLX as a backend option for model scoring and serving. The platform support documentation in the repository covers macOS-specific detection details.
Yes. The custom models guide in the repository covers adding models locally without a rebuild. You can also contribute new models to the built-in catalog via a pull request; the catalog is community-maintained and grows through contributions from users with new hardware and model combinations.
Alternative to
Grok Bot
Self-hosted AI agent: persistent memory, multi-channel chat, model choice.
Alternative to
Together AI
High-throughput LLM inference server with OpenAI-compatible API
Alternative to
OpenAI API
Self-hosted AI runtime: any model, any modality, any hardware.
Phase-guarded workflow layer for AI coding agents, with eval built in
Alternative to
Conductor
Desktop canvas for persistent multi-agent AI teams, Windows-only
Alternative to
Context7
Convert docs URLs into Markdown bundles for AI coding agents
Run llmfit on your own server
Self-host llmfit on a Hostinger VPS.
Deploy on HostingerFeatured
Alternative to
Superhuman
AI email assistant that drafts replies and clears your inbox
Alternative toGrok Bot
AI teammates you own: your keys, your model, your machine.
Alternative toClaude Managed Agents
Open-source agent deployment API. Self-host or use Omnara Cloud.
The projects we add and every new guide. Free, unsubscribe any time.
Claude Code to 
OpenCode
Jira to 
Windshift
Notion to 
AppFlowy
Slack to 
Mattermost
Linear to 
GitLab
Wispr Flow to 
Handy
CapCut to 
Concat
Grok Bot to 
rakazo
Superhuman to 
Inbox Zero
Airtable to 
AITable
Muse to 
OpenClaw
GitHub Copilot to 
Cline
Datadog to 
Netdata
Microsoft Teams to 
Rocket.Chat
Pinecone to 
Qdrant
Zapier to 
n8n
Devin AI to 
OpenHands
Firebase to 
Supabase
1Password to 
KeePassXCConfluence to 
Docmost
Contentful to 
Strapi
Adobe Premiere Pro to 
Shotcut
ChatGPT to 
AnythingLLM
LangSmith to 
Langfuse
