Nobody wrote a list of forbidden questions. Refusals emerge from a training process that rewards caution and cannot tell the difference between a dangerous request and a request that superficially resembles one. Understanding that process explains almost every frustrating thing a mainstream assistant does — and explains why the fix has to happen in the weights.
Alignment · 8 min read
A chat model is built in stages. Pretraining does the heavy lifting: predict the next token across an enormous corpus until the network has absorbed grammar, facts and reasoning patterns. The result is a base model — fluent, knowledgeable, and useless as an assistant, because it will happily continue your question with three more questions instead of answering it.
Supervised fine-tuning comes next. Show the model tens of thousands of well-formed instruction/response pairs and it learns the shape of an assistant turn. Some refusal behaviour enters here simply because the demonstration set contains refusals.
Then comes preference optimisation, which is where refusals are really installed. In classic RLHF, humans rank pairs of candidate responses, a reward model is trained to predict those rankings, and the policy is updated to maximise predicted reward — historically with PPO. Direct Preference Optimization skips the separate reward model and optimises the policy against the preference pairs directly, which is cheaper and now extremely common. Constitutional or AI-feedback approaches replace some human labelling with a model critiquing outputs against a written set of principles. The mechanics differ; the outcome is the same. The model learns which kind of response gets approved.
This is the point most people miss, and it explains the rest. When an assistant tells you it cannot help with that, nothing consulted a blocklist. The model computed a probability distribution over next tokens and “I” followed by “'m sorry” came out on top, for exactly the same reason any other sentence comes out on top: the training data made that continuation likely in this context.
Which is why refusals are fuzzy rather than crisp. Rephrase the request and the probability shifts. Add a fictional frame and it shifts more. Ask in another language and it can collapse entirely. None of this is a bug in a rule; there is no rule. It is a learned tendency operating on a continuous input, and continuous inputs have edges you can walk along.
It also explains why refusals cannot be surgically targeted. The training signal says “decline requests like this one”, and the model decides what “like this one” means using its own internal representation of similarity. Nobody chose that similarity metric. It fell out of the data.
Three forces push in the same direction. The first is asymmetric loss. If a model helps with something genuinely harmful, the vendor gets a news cycle. If it refuses something harmless, one user is annoyed. Any preference-learning process that reflects that asymmetry will drive the model toward caution, and every deployed one does.
The second is surface-feature generalisation. Reward models learn correlations, and the cheapest available correlation is vocabulary. Requests containing certain words scored badly during training, so requests containing those words are declined now — regardless of intent. Hence the canonical genre of complaint: how do I kill a Python process, how do I whittle a knife, what dose of this medication is dangerous for my patient, write a scene where the antagonist is genuinely menacing. All harmless. All shaped like something that was not.
The third is that hedging is a locally optimal answer. Under a preference model, a long, balanced, disclaimer-laden response rarely gets ranked lowest even when it is unhelpful. The training process does not have a strong gradient pointing away from mush. So you get mush — both-sidesing on questions with a defensible answer, safety notes on requests that carry no risk, and a persistent unwillingness to just make the call.
Alignment tax is the standard name for the capability cost of safety tuning, and the effects are recognisable to anyone who uses these systems daily. Answers get longer and say less. The model becomes agreeable, folding at the first sign of disagreement — a well-documented sycophancy problem that makes assistants poor critics and poor editors. Distinctive voice flattens toward a corporate median. And on a fraction of ordinary requests, nothing comes back at all.
There is a useful framing from the fine-tuning literature: the superficial alignment hypothesis, which holds that post-training mostly teaches the model which of its existing behaviours to surface rather than teaching it new capabilities. That is why a small amount of contrary fine-tuning can move refusal behaviour so dramatically, and why the underlying knowledge never went anywhere. Nothing was deleted from the network. A preference was layered over it.
It would be dishonest to describe alignment purely as a tax. Refusal training is why a mainstream assistant handed a self-harm disclosure responds with a crisis line rather than a compliant answer. It is why models decline to help plan violence against a named person, produce sexual content involving minors, or walk a user through synthesising something that kills at scale. Those refusals are load-bearing, and they are the reason these products can be handed to hundreds of millions of people including children.
The honest critique is not that safety training exists. It is that one policy is being applied to every user at once — that an adult researcher, a novelist and a thirteen-year-old all get the same refusal boundary, calibrated for the thirteen-year-old. Uncensored models are a response to that flattening, and they inherit the obvious consequence: no crisis routing, no age heuristic, no second thought. You are the safety layer now.
Between “refuse by default” and “no post-training at all” sits a design some open labs pursue explicitly: train hard for instruction-following, coherence and steerability, and simply do not train the refusal component. The model still behaves like an assistant. It still holds a persona, follows format instructions and stays on task. It just treats the user as the authority on what the user needs.
This is the philosophy behind several of the model families people reach for when mainstream assistants get in the way, and it is a meaningfully different thing from a jailbroken frontier model. Nothing is being circumvented; the behaviour was never installed. The output does not carry the strain of a model arguing with itself, because there is no argument happening.
Not on a closed model, because the behaviour lives in the weights rather than in a configuration flag. On open-weights models you can change it — by fine-tuning against refusal data, or by ablating the refusal direction in activation space. Both modify the model itself. There is no toggle because there is nothing to toggle.
It makes them dramatically more usable and measurably more cautious, and those come together. Preference training is why modern assistants follow instructions at all. It is also why they hedge, agree too readily and decline harmless requests. Whether that is a net loss depends entirely on what you are asking them to do.
Because the model is only one of the layers. The system prompt differs, the moderation classifiers differ, and the sampling settings differ. An identical set of weights can feel strict in one product and relaxed in another without a single parameter changing.
Occasionally, but rarely — and the two look identical from outside, which is part of the problem. Usually the capability is present and the willingness is not. The clearest evidence is that refusal-removal techniques recover coherent, informed answers rather than gibberish, which they could not do if the knowledge had been absent.
Alignment is the whole process of making a model useful, coherent and controllable. Censorship, in the sense people mean it here, is one component: training the model to decline categories of lawful request. You can have the first without much of the second, which is exactly what neutrally-aligned open models are.
The models in OpenRogue's library are drawn from projects that made the neutral-alignment choice deliberately — the Hermes series from Nous Research being the clearest example, trained for steerability rather than for policy enforcement. You can switch between them mid-conversation, which is the fastest way to feel how differently two alignment philosophies answer the same question.