heretic

36K stars
AGPL-3.0
Active this week
Last commit 1 day ago

Last verified Oct 20, 2026

Automate the removal of safety restrictions from local language models using directional ablation, with no ML expertise required. AGPL-3.0, Python.

Visit website GitHub36K Deploy on Hostinger

Read the full breakdownWho it is for, how it works, trade-offs, the quick start and the FAQ

who it's for

Who heretic is for#

ML researchers studying safety alignment mechanisms

Heretic's interpretability extras let you visualize residual vectors, compute cosine similarities across layers, and generate layer-by-layer animations of how harmful vs. harmless prompts differ in representation. This supports empirical work on where and how safety alignment is encoded in transformers without requiring custom tooling.

Skip if:

Skip if your research focuses on building or improving alignment rather than analyzing it, or if you need a tool with formal academic support and published peer review rather than a community project.

Developers building local AI applications needing unrestricted outputs

If your application requires a locally hosted LLM that responds to any prompt without content refusals, Heretic automates the process of producing that model from a Hugging Face checkpoint. The entire workflow is a pip install and one command; no knowledge of transformer architecture is needed.

Skip if:

Skip if your deployment targets a cloud API (commercial providers will not run user-modified model weights), or if your use case is already served by an existing heretic-tagged model on Hugging Face.

Self-hosted AI enthusiasts evaluating open model quality after abliteration

Heretic's Hugging Face integration and interactive chat session after each run make it practical for comparing how different base models respond once safety filters are removed. The built-in benchmark runner (MMLU, GSM8K) provides a consistent quality signal across abliteration runs on different model families.

Skip if:

Skip if you prefer pre-built model images rather than running a 20-to-30-minute abliteration job locally. The Hugging Face model hub already has over 5,000 heretic-tagged models ready to download.

LLM security engineers running red-team baseline evaluations

Security and red-teaming engineers can use Heretic to produce controlled uncensored variants of frontier open models, providing a baseline comparison point for evaluating how effectively safety alignment prevents undesired outputs before and after abliteration.

Skip if:

Skip if your organization prohibits running uncensored models internally, or if you need commercial support and liability coverage for the tooling used in security evaluations.

the problem

The problem it solves#

Running a local language model with no content restrictions has historically required deep expertise in transformer internals. Manual abliteration involves identifying residual directions across layers, selecting ablation weight kernels, and tuning parameters by hand: work accessible only to ML researchers with specialized background.

At the same time, commercial API services enforce content policies that block entire categories of queries outright. Teams that need unrestricted outputs from a locally hosted model had no automated path: they either engaged expert ML contractors or accepted the restrictions of closed platforms with per-token pricing and no self-hosting option.

how heretic solves it

How it solves it#

Automatic ablation parameter optimization

Heretic uses a TPE-based optimizer (Optuna) to find ablation parameters without manual tuning. It co-minimizes refusal rate and KL divergence from the original model simultaneously, so the decensored output retains as much original intelligence as possible. On a gemma-3-12b-it benchmark, Heretic reached the same refusal suppression as expert-made abliterations at a KL divergence of 0.16, compared to 0.45 and 1.04 for two competing manually tuned alternatives.

Flexible ablation weight kernel with float direction index

Unlike fixed-weight abliteration approaches, Heretic applies a shape-optimizable weight kernel over transformer layers, with separate ablation parameters for attention out-projection and MLP down-projection. The residual direction index is a float rather than an integer, enabling interpolated directions that extend beyond what difference-of-means computation alone identifies, unlocking a larger optimization space.

Broad model architecture support

Supports most dense transformer models, many multimodal models, several MoE architectures, and hybrid models like Qwen3.5. Load a model from Hugging Face by name and Heretic handles the rest. Pure state-space models and certain research architectures are not yet supported out of the box.

Built-in quantization via bitsandbytes

Set the quantization option to bnb_4bit to run Heretic on models that would otherwise exceed available VRAM. Quantization is applied during the decensoring process itself, making it practical to abliterate large models on consumer-grade GPUs without needing a separate quantization step.

Post-run evaluation, benchmarking, and upload

After decensoring, Heretic offers to save the model locally, upload it directly to Hugging Face, launch an interactive chat session to evaluate quality, or run standard benchmarks (MMLU, GSM8K). All options are available in any combination from a single menu prompt at the end of each run.

Research interpretability extras

The optional research extra adds tools for studying model internals: PaCMAP-based visualizations of residual vectors across transformer layers, animated GIFs showing how residuals evolve layer by layer, and a quantitative table of residual geometry metrics including cosine similarity, L2 norms, and silhouette coefficients per layer.

strengths · trade-offs

Strengths and trade-offs#

Strengths

Fully automatic: no ML expertise requiredAnyone who can run a command-line program can use Heretic. The optimization loop finds high-quality ablation parameters without human guidance. This is the core differentiator from prior art tools like abliterator.py and AutoAbliteration, which require manual parameter selection and iterative evaluation by someone who understands transformer internals.
Lower capability damage than manual abliterationsBenchmarks on gemma-3-12b-it showed Heretic's automatically generated abliteration achieved 3/100 refusals (matching expert hand-tuned results) at a KL divergence of 0.16, compared to 0.45 and 1.04 for two competing manually produced abliterations. Lower KL divergence means less damage to the model's reasoning and generation quality.
Over 5,000 community-published models on Hugging FaceThe Heretic community has published more than 5,000 models to Hugging Face tagged with the heretic topic. This provides an immediately accessible library of pre-abliterated models for common architectures, so users who cannot run Heretic locally can still access its output without running the tool themselves.
Written from scratch with a research orientationHeretic does not reuse code from any prior abliteration project. Its research extras provide quantitative metrics (residual geometry tables, PaCMAP projections) that support interpretability work, making it useful beyond censorship removal for researchers studying how safety alignment is encoded in transformer layers.

Trade-offs

-Requires consumer GPU hardware with significant VRAMAbliterating a 4B model on an RTX 3090 takes 20 to 30 minutes. Larger models need substantially more VRAM. While bitsandbytes quantization reduces requirements, very large models are impractical on anything below a high-end GPU setup. CPU-only runs are not documented in the current README.
-AGPL-3.0 requires source disclosure for hosted servicesAGPL-3.0 is a copyleft license. If you run a modified version of Heretic as a network service (for example, an API endpoint that accepts models and returns decensored versions), you must release your modifications under the same license. Teams wanting to build a proprietary service on top of Heretic need to evaluate this constraint carefully before integrating it.
-Pure state-space models and some architectures not yet supportedHeretic explicitly states that pure state-space models and certain other research architectures are not supported out of the box. If your target model falls into this category, you will need a different abliteration approach or wait for future Heretic support.

versus alternatives

heretic vs alternatives#

Heretic vs AutoAbliteration

AutoAbliteration is Maxime Labonne's approach to automated abliteration, published as a Hugging Face Space and script. Both tools aim to reduce manual effort in the abliteration workflow. Heretic's key advantage is its TPE-based parameter optimizer, which explores the full ablation weight kernel shape automatically. AutoAbliteration applies simpler fixed-weight ablation and leaves parameter selection to the user. Heretic benchmarks show lower KL divergence at the same refusal suppression rate compared to Labonne's own models produced without the optimizer.

AutoAbliteration is a reasonable starting point if you want a lightweight script without the Optuna dependency. Heretic is the better choice when output quality matters: the automated search finds better parameter configurations than fixed defaults across most model architectures.

Heretic vs abliterator.py

abliterator.py (by FailSpy) is the most widely referenced manual abliteration script in the community. It requires the user to select which layers to ablate, which direction indices to use, and what weight to apply. Heretic replaces that entire manual process with an optimization loop. For users without transformer expertise, abliterator.py is effectively inaccessible; Heretic is not.

abliterator.py remains the better choice for researchers who need precise manual control over which specific layers and directions are modified. Heretic's optimizer may not always explore the exact parameter combination a researcher is investigating. For production use cases where output quality is the primary goal rather than fine-grained control, Heretic's automatic approach consistently outperforms manual configurations in benchmark comparisons.

install · quick start

Quick start#

bash

Install Heretic via pip into a Python 3.10+ environment with PyTorch 2.2+ configured for your hardware.
```bash
pip install -U heretic-llm
heretic Qwen/Qwen3-4B-Instruct-2507
```

tech stack · detected from GitHub

What it's built on#

Languages: Python

frequently asked

FAQ#

What is abliteration and how does it differ from fine-tuning?

Abliteration is a post-training intervention that modifies a model's weight matrices to suppress a specific direction in residual space, without any gradient updates or new training data. Fine-tuning requires labeled examples and a full training run; abliteration operates directly on the saved model weights in a single forward-pass analysis step. Heretic automates the abliteration process using parameter optimization, making it faster and less expertise-dependent than either manual abliteration or fine-tuning.

What hardware do I need to run Heretic?

Heretic requires a CUDA-capable GPU with enough VRAM to hold the target model. On an RTX 3090 (24GB VRAM), a 4B-parameter model takes 20 to 30 minutes to process. Larger models require proportionally more VRAM, but bitsandbytes 4-bit quantization is supported and reduces requirements significantly. Some model configurations may require PyTorch 2.6+ for features like MXFP4 quantization. CPU-only execution is not documented in the current README.

Can I use Heretic with models that are not on Hugging Face?

Yes. The heretic CLI accepts any model path that the underlying transformers library can load, including local directories in the standard Hugging Face safetensors format. You do not need a Hugging Face account to abliterate a local model checkpoint; an account is only needed if you want to upload the result to the Hugging Face hub.

What does the AGPL-3.0 license mean for my use of Heretic?

AGPL-3.0 is a copyleft license. Running Heretic locally for personal or internal research is unrestricted. If you modify Heretic and run it as a network service accessible to others (such as an API endpoint), you must release your modifications under AGPL-3.0 as well. The license applies to Heretic itself, not to the models you produce with it; the decensored model output is governed by the license of the base model you started from.

How does Heretic compare to other abliteration tools like abliterator.py or ErisForge?

Heretic's main advantage is automatic parameter optimization via Optuna's TPE algorithm, which selects ablation weights without manual tuning. In benchmarks on gemma-3-12b-it, Heretic produced the same refusal suppression as manually tuned alternatives (3/100 refusals) at significantly lower KL divergence (0.16 vs 0.45 to 1.04), indicating less damage to the model's capabilities. Heretic was also written from scratch and does not share code with any prior abliteration project. The tradeoff is that Heretic's optimizer makes choices automatically, which may not suit researchers who need precise manual control over which layers and directions are modified.

also worth a look

Similar open-source tools#

Hermes Agent

Featured

Alternative toGrok Bot

Self-hosted AI agent: persistent memory, multi-channel chat, model choice.

255,514 GitHub stars
Licence MIT

Vllm

Alternative toTogether AI

High-throughput LLM inference server with OpenAI-compatible API

94,441 GitHub stars
Licence Apache-2.0

LocalAI

Alternative toOpenAI API

Self-hosted AI runtime: any model, any modality, any hardware.

49,798 GitHub stars
Licence MIT

Comet

InAI & Machine Learning

Phase-guarded workflow layer for AI coding agents, with eval built in

3,206 GitHub stars
Licence MIT

Orgtree

Alternative toConductor

Desktop canvas for persistent multi-agent AI teams, Windows-only

168 GitHub stars
Licence MIT

Agent Cache Web

Alternative toContext7

Convert docs URLs into Markdown bundles for AI coding agents

11 GitHub stars
Licence MIT

by category

Open source tools in these categories#

404

Run heretic on your own server

Self-host heretic on a Hostinger VPS.

Deploy on Hostinger

Featured

rakazo

Alternative toGrok Bot

AI teammates you own: your keys, your model, your machine.

4,904 GitHub stars
Licence Apache-2.0
#AI Agents

Omnara

Alternative toClaude Managed Agents

Open-source agent deployment API. Self-host or use Omnara Cloud.

2,957 GitHub stars
Licence Apache-2.0
#AI Agents

Inbox Zero

Alternative toSuperhuman

AI email assistant that drafts replies and clears your inbox

12,712 GitHub stars
Licence AGPL-3.0
#Email

Feature your project

The projects we add and every new guide. Free, unsubscribe any time.

Who heretic is for#

ML researchers studying safety alignment mechanisms

Skip if:

Developers building local AI applications needing unrestricted outputs

Skip if:

Self-hosted AI enthusiasts evaluating open model quality after abliteration

Skip if:

LLM security engineers running red-team baseline evaluations

Skip if:

The problem it solves#

How it solves it#

Automatic ablation parameter optimization

Flexible ablation weight kernel with float direction index

Broad model architecture support

Built-in quantization via bitsandbytes

Post-run evaluation, benchmarking, and upload

Research interpretability extras

Strengths and trade-offs#

Strengths

Trade-offs

heretic vs alternatives#

Heretic vs AutoAbliteration

Heretic vs abliterator.py

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

AI & Machine Learning

Developer Tools

LLMOps & AI Tooling

Find open source alternatives to paid software

Who heretic is for#

ML researchers studying safety alignment mechanisms

Skip if:

Developers building local AI applications needing unrestricted outputs

Skip if:

Self-hosted AI enthusiasts evaluating open model quality after abliteration

Skip if:

LLM security engineers running red-team baseline evaluations

Skip if:

The problem it solves#

How it solves it#

Automatic ablation parameter optimization

Flexible ablation weight kernel with float direction index

Broad model architecture support

Built-in quantization via bitsandbytes

Post-run evaluation, benchmarking, and upload

Research interpretability extras

Strengths and trade-offs#

Strengths

Trade-offs

heretic vs alternatives#

Heretic vs AutoAbliteration

Heretic vs abliterator.py

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

AI & Machine Learning

Developer Tools

LLMOps & AI Tooling

Find open source alternatives to paid software