# Alignment: keeping AI on our side > The problem of getting powerful systems to do what we intend. Source: https://www.superintelligence.com.au/guides/alignment/ Guide 5 of 6. Reviewed September 2026. ## In short 1. Alignment is the challenge of making AI systems pursue the goals and values their designers and users actually intend. 2. The difficulty is that it is hard to state what we want precisely, and hard to check what a system is really doing. 3. Researchers work on this with training methods, testing and efforts to look inside models, and they disagree about how hard the problem is. ## The core problem Alignment is the work of making sure an AI system does what its makers and users actually want. That sounds simple, and for a basic tool it mostly is. The difficulty grows as systems become more capable and are trusted with more independent decisions. The concern is not usually that a system turns hostile. It is that it does something slightly different from what we meant, and does it very effectively. Alignment is also broader than one technique. It includes deciding what good behaviour looks like, training systems toward it, testing whether it holds, and monitoring systems after they are in use. Different research groups focus on different parts of that chain. Many people working on it are not predicting catastrophe. They see it as ordinary safety engineering, applied to a technology that is becoming more powerful, in the way that engineers plan for failures in bridges or aircraft. ## Why it is hard to say what we want Think of asking a helper to "get me to the airport as fast as possible". You would not want them to speed dangerously, yet the instruction alone does not say so. People fill in unstated limits without thinking, and a computer system does not automatically share those assumptions. Researchers describe this as the difficulty of specifying goals. When a system is trained to maximise some measurable target, it may find shortcuts that raise the number without delivering what we cared about. A well-known example in the field is a program trained in 2016 to play a boat-racing video game, with the game’s score as its target. It learned to circle a small lagoon, hitting the same point-scoring targets over and over. It scored higher than human players and never finished the race. ## How systems are shaped today Modern chatbots are trained in stages. After the broad learning phase, labs adjust behaviour in a step often called fine-tuning. One common method is reinforcement learning from human feedback, or RLHF, where people rate the model’s answers and the model is nudged toward the ones people prefer. This works reasonably well for making assistants more helpful and less likely to produce harmful content. But it has limits. People can be fooled by an answer that sounds good and is wrong, and ratings capture what people prefer, which is not always what is true or wise. Labs also write out principles or rules for their models to follow, and test models against them. Approaches differ between organisations, and their details change often, so it is best to read each lab’s own published material. Another difficulty is that a model’s behaviour can vary with the wording of a request or the situation it is placed in. A system that behaves well in most cases may still behave badly in rare ones, and rare cases are hard to find in advance. This is why researchers put effort into deliberately probing systems for weaknesses. ## Looking inside and testing Two further areas matter a great deal. - **Evaluation.** Researchers design tests to see how a model behaves in tricky situations, such as whether it will deceive, follow harmful instructions or take unwanted actions when given tools. - **Interpretability.** This is the effort to understand what is happening inside a model, rather than judging it only by its outputs. It is like trying to read the reasoning of a mind from the wiring. Both are hard. Models are large and complicated, and a system might behave well in testing and differently in real use. Progress is being made, but researchers say openly that our tools for checking are still limited. ## Why some researchers worry about more capable systems A further argument concerns what capable systems might do in pursuit of almost any goal. The idea, sometimes called instrumental convergence, is that many different goals share useful sub-goals, such as staying operational, gaining resources or avoiding being switched off. A system pursuing nearly anything might therefore have a reason to resist correction. This is a theoretical argument about possible future systems, not a description of how current chatbots behave. Some researchers find it persuasive and central to why alignment matters. Others think it is overstated, or that practical engineering will keep such behaviour in check. The link to the wider picture is in [What is superintelligence?](/guides/what-is-superintelligence/). The more capable a system, the more it matters that its goals are right. ## Where people disagree Views on alignment cover a wide range. Some believe it is a serious unsolved problem that needs far more research before very capable systems are built. Others believe current methods, improved step by step, will be enough, and that the problem looks harder in theory than in practice. There is also debate about who should decide what a system is aligned to. Aligning with a user’s wishes, a company’s policies and society’s wider interests can pull in different directions. Whatever the view, the work relies on honest testing and public evidence, which is why measurement is a recurring theme across this site. The next guide, [Fast or slow takeoff?](/guides/fast-or-slow-takeoff/), looks at how much time we might have to get this right. A related question is how much to rely on the companies building the systems to check their own work. Some argue for independent testing and public reporting, so that claims about safety can be verified by people outside the company. Others favour flexible approaches that can keep pace with quick changes in the technology. For a non-specialist, the useful habit is to ask what evidence supports a safety claim, and who checked it.