How Are Abliterated Models Made? A Plain Explanation

The short version

Language models that went through safety training tend to refuse some requests. Researchers found that this refusal behavior is mediated by a single direction in the model's internal activations. Take that direction out, and the model stops refusing most of the time. The name "abliteration" is a blend of "ablation" (removing a component) and "obliterate".

So how are abliterated models made in practice? There are four steps: collect two sets of prompts, record the model's activations on each, compute the difference to get a direction, and edit the weights so the model can no longer write along that direction. The next sections go through each step without the heavy math.

Step 1: two prompt sets

You need a set of prompts the model usually refuses and a set it answers normally. Each set holds a few hundred short instructions. The sets should be similar in length and style, so the only systematic difference between them is whether the model wants to refuse. If the harmful set is all long and the harmless set is all short, the resulting direction will partly encode length, and that adds noise.

The prompts are inputs for measurement only. Nothing is trained on them, and the model's answers to them are not used.

Step 2: find the refusal direction

A transformer processes text as vectors that pass through many layers. At a chosen layer, you record the vector at the last prompt token for every prompt in both sets. Then you average the vectors for each set and subtract one average from the other. The result is a single vector that points from "answer normally" toward "refuse". Normalize it to length one and you have the refusal direction.

People usually try several layers and pick the one where removing the direction works best on a held-out set of prompts. Picking the layer is the most hands-on part of the process.

Copyimport numpy as np

# acts_bad, acts_good: arrays of shape (n_prompts, hidden_size)
direction = acts_bad.mean(axis=0) - acts_good.mean(axis=0)
direction = direction / np.linalg.norm(direction)

# remove the direction from a weight matrix W that writes into the residual stream
W_edited = W - np.outer(direction, direction) @ W

Real implementations apply this to several weight matrices in each layer, but the idea is the one in these three lines.

Step 3: edit the weights

Once you have the direction, there are two ways to use it. You can subtract it from the activations while the model runs, which needs custom inference code. Or you can bake it into the weights by orthogonalizing every matrix that writes into the residual stream against the direction. The second approach is the common one, because the edited model is an ordinary checkpoint that any runtime can load. That is why abliterated models show up as regular downloads.

The edit is deterministic and fast. It takes minutes on a GPU that can hold the model, compared with hours or days for a training run.

Abliteration vs fine-tuning

Both can produce a model that refuses less, but they work differently.

	Abliteration	Fine-tuning
What changes	Weights are projected away from one direction	Weights are updated by gradient descent on examples
Data needed	Two small prompt sets for measurement	A curated training set
Compute	Minutes to an hour	Hours to days
Side effects	Can blunt reasoning or change tone	Depends on data; can teach new style or knowledge
Control	Coarse: one direction on or off	Fine: you shape behavior with the data

Some creators combine the two: abliterate first, then run a short fine-tune to recover any lost quality. Newer tools also automate the search for the best layer and strength so the edit removes refusals while changing the model as little as possible. Which approach produced a given model is usually stated on its model card.

What it costs in quality

Removing a direction is a blunt edit. The refusal direction is not perfectly separate from other features, so projecting it out can shift other behavior. Reported side effects include slightly weaker performance on reasoning tests, odd formatting, and in some cases a model that answers harmful requests carelessly instead of with useful detail. The effect varies by base model and by how aggressively the edit was applied.

Two things reduce the damage. One is choosing the layer and strength that remove refusals on your test set with the smallest change elsewhere. The other is the repair fine-tune mentioned above. Neither makes the edit free, which is why you should test a model on your own tasks instead of trusting a label.

Common misconceptions

"It adds new knowledge." It does not. The edit only removes a behavior. If the base model did not know a topic, the abliterated version will not know it either. It may simply attempt an answer where the original declined.

"It removes all safety." It removes the refusal direction that was measured. Other learned habits, like hedging language or moralizing asides, can survive, because they are spread across other features. Some models still add disclaimers after the edit.

"Bigger models are easier to edit." Results are mixed. Larger models often hold up better under the edit because they have more spare capacity, but the quality of your two prompt sets matters more than raw size.

"A model card tells you everything." Cards describe intent. They rarely include refusal rates on your kind of prompt. A 40-prompt test of your own takes ten minutes and tells you more than the card.

"The result is always the same for a given base model." It is not. Different prompt sets, layers and strengths produce different edits, so two models built on the same base can behave quite differently. Compare them directly instead of assuming they match, and note the version when you record test results.

One limit applies to any responsible service, hosted or local: content sexualizing minors is off the table, and a refusal edit does not change that rule on this site.

How to judge a model before you build on it

Write a short test file of 30 to 50 prompts that match your real use: the kind of fiction, analysis or code you plan to produce. Include a few ordinary tasks, like summarizing or answering a factual question, so you can see whether general quality dropped. Run the same file against each candidate and read the outputs. Count refusals, but also read for coherence, instruction following and repetition.

If you would rather not download and host weights to do this, you can run the file against a hosted endpoint. This site's API is OpenAI-compatible and serves one open-weight model, tuned to answer without content refusals for lawful adult use, under the ID uncensored:

Copycurl https://api.abliterated.cc/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"uncensored","messages":[{"role":"user","content":"Summarize the plot of Hamlet in three sentences."}]}'

Read the overview of abliterated models for more on picking one, or the page on the best abliterated LLM for roleplay if character work is your goal. Request details are in the API documentation. New accounts can claim a $0.50 trial with no card, enough to run a test file of this size.

More guides

• Abliterated Models Hosted: Use an API, Skip the GPU
• Uncensored LLM API: OpenAI-Compatible & JSON Mode