{"updated":"2026-10-07","attribution":"Benchmark data from Epoch AI (CC BY) and official leaderboards.","scoreboard":"| Benchmark | Best score | Record held by | Since | Change in 12 months | Compared with people |\n| --- | --- | --- | --- | --- | --- |\n| [GPQA Diamond](https://www.superintelligence.com.au/benchmarks/gpqa-diamond/) | 96% | GPT-6 Astra (max) | Q3 2026 | +3 pts in 12 months | Experts in their own field: 81% |\n| [SWE-bench Verified](https://www.superintelligence.com.au/benchmarks/swe-bench-verified/) | 84% | Claude Opus 4.7 (max) | Q2 2026 | +7 pts in 12 months | There is no published human score. |\n| [FrontierMath](https://www.superintelligence.com.au/benchmarks/frontiermath/) | 94% | GPT-6.1 Sol (max) | Q3 2026 | +20 pts in 12 months | No single human baseline exists. |\n| [Humanity’s Last Exam](https://www.superintelligence.com.au/benchmarks/humanitys-last-exam/) | 55% | GPT-6 Astra (unknown thinking) | Q3 2026 | +17 pts in 12 months | There is no overall human score. |\n| [ARC-AGI-2](https://www.superintelligence.com.au/benchmarks/arc-agi-2/) | 95% | GPT-6 Astra (max) | Q3 2026 | +41 pts in 12 months | The test’s creators had more than 400 members of the public attempt the puzzles, and every puzzle was solved by at least two of them. |\n\nEach score is the best any model had reached by the end of the quarter, by model release date. Ask for one benchmark by name for its history, leaders and caveats.","docs":[{"path":"/guides/what-is-superintelligence/","kind":"guide","title":"What is superintelligence?","summary":"What people mean by the word, and why it’s taken seriously.","aliases":[],"markdown":"# What is superintelligence?\n\n> What people mean by the word, and why it’s taken seriously.\n\nSource: https://www.superintelligence.com.au/guides/what-is-superintelligence/\nGuide 1 of 6. Reviewed September 2026.\n\n## In short\n\n1. Superintelligence means an intellect that greatly outperforms the best humans across almost every area that matters.\n2. No such system exists today, and researchers disagree about whether, when and how it might arrive.\n3. The topic is taken seriously because leading AI labs and many researchers treat it as a possible outcome worth preparing for.\n\n## A simple definition\n\nSuperintelligence is a word for an intelligence that is far better than the best human minds at nearly everything that matters. That includes science, planning, persuasion, engineering and social understanding. It is not a claim about one clever trick, like playing chess, which computers have done better than people for decades.\n\nThe word describes a level of ability, not a particular technology. It could in principle be a single program, a network of programs working together, or something we have not imagined yet.\n\n## Where the idea comes from\n\nThe idea is older than modern AI. The mathematician I. J. Good wrote in 1965 that a machine smarter than any person could design even better machines, leaving human intelligence far behind. He called this an \"intelligence explosion\".\n\nThe word \"superintelligence\" reached a wide audience through Nick Bostrom’s 2014 book of the same name. Bostrom set out how such a system might arise, what forms it could take, and why the question of control would matter. Since then the topic has moved from philosophy seminars into the plans of well-funded companies.\n\nGood’s point was simple. If intelligence is the ability that produces new technology, then a machine with more of it might produce new technology faster than we can follow. Whether that reasoning holds up in practice is still argued over, but it explains why the idea has lasted.\n\nIt is also worth separating the idea from science fiction. Films tend to show machines with human motives, such as revenge or ambition. Researchers who study the topic are usually concerned with something duller and harder: a system pursuing a goal competently while we fail to notice that the goal is slightly wrong.\n\n## What it is not\n\nIt helps to clear away a few common pictures. Superintelligence is not the same as a very good chatbot, and it is not a robot with a human face. It also does not have to be conscious, angry or evil to matter.\n\n- **Not today’s AI.** Current systems are impressive at many tasks and still make basic mistakes. Nobody serious claims they are superintelligent.\n- **Not a prediction.** Talking about it does not mean it is certain, or close.\n- **Not one thing.** People use the word for quite different scenarios, from a single system to many cooperating ones.\n\nIf you want the vocabulary sorted out, the next guide, [AI, AGI and ASI: the difference](/guides/ai-agi-and-asi/), separates the three terms people mix up most.\n\nA helpful test is to ask what would have to be true. A superintelligent system would need to outperform skilled people in scientific research, in long-term planning, in writing and in understanding other people, and not merely in one narrow contest. That is a very high bar, which is part of why the word is used carefully.\n\nAnother point is that ability is not the same as motive. A system can be extremely capable and still simply do what it is asked. The concern is about what happens when what it is asked, and what we meant, quietly differ.\n\n## Why people take it seriously\n\nThere are a few reasons the idea gets attention. AI systems have improved quickly on many tests over the past decade, and the largest companies building them say openly that they are aiming at systems that match or exceed people across a wide range of work. Several prominent researchers, including some who helped build the field, have said the risks deserve serious effort.\n\nAnother reason is the shape of the problem. If a system became far more capable than us, we might have only limited ability to correct mistakes afterwards. That makes preparation before the fact more valuable than repair after it. The [alignment guide](/guides/alignment/) covers what that preparation involves.\n\n## Where researchers disagree\n\nNot everyone is persuaded. Some researchers think current methods will stall well short of human-level ability, and that talk of superintelligence distracts from harms happening now, such as errors, bias and misuse. Others think progress will continue and that the far end of the range is reachable.\n\nThere is also disagreement about how quickly things could move. Some expect a gradual climb over many years, while others think a sudden jump is possible once systems can help improve themselves. The guide on [fast or slow takeoff](/guides/fast-or-slow-takeoff/) sets out both views fairly.\n\nNobody knows the answer, and this site does not offer a date. Anyone who gives you a confident one is going beyond the evidence.\n\n## How to think about it\n\nA sensible way to hold the idea is as a question with high stakes and low certainty. It does not call for panic, and it does not deserve to be waved away either. What you can do is learn how progress is measured, what the main arguments are, and where the experts differ.\n\nThe rest of this site is built for that. The [benchmark explorer](/benchmarks/) shows how leading systems perform on hard tests, with live numbers rather than a snapshot in an article. A good next step is the guide on [how AI progress is measured](/guides/how-ai-progress-is-measured/), which explains what those tests can and cannot tell you.\n\nIt can help to keep two questions apart. The first is whether such a system is possible at all, which is a question about science and engineering. The second is what we should do now given that we are uncertain, which is a question about judgement and risk. People can agree on one and disagree on the other.\n\nYou do not need a technical background to follow the debate. You need patience with unfamiliar terms, and a willingness to ask what evidence sits behind a claim.\n"},{"path":"/guides/ai-agi-and-asi/","kind":"guide","title":"AI, AGI and ASI: the difference","summary":"Three terms that get mixed up constantly, sorted in one page.","aliases":[],"markdown":"# AI, AGI and ASI: the difference\n\n> Three terms that get mixed up constantly, sorted in one page.\n\nSource: https://www.superintelligence.com.au/guides/ai-agi-and-asi/\nGuide 2 of 6. Reviewed September 2026.\n\n## In short\n\n1. AI is the broad label for software that performs tasks that normally need human intelligence, and it covers everything from spam filters to chatbots.\n2. AGI usually means a system that can do most intellectual work at about the level of a capable person, but there is no single agreed definition.\n3. ASI means a system well beyond the best humans, and it is a separate and higher bar than AGI.\n\n## Three labels, three levels\n\nPeople often use AI, AGI and ASI as if they meant the same thing. They do not. A useful way to remember them is as three steps on a ladder: tools that do particular jobs, then systems that can do most jobs a person can, then systems that go far beyond us.\n\nEach step is a matter of degree, and the edges are blurry. Even so, keeping the three apart makes most news stories easier to read.\n\n## AI: the everyday word\n\nArtificial intelligence, or AI, is the widest term. It covers any software that does something we would normally expect to need human thinking, such as recognising speech, recommending a film, translating a sentence or writing an email.\n\nMost AI in use today is what researchers sometimes call narrow. It is built or trained for a set of tasks and works well inside them. Modern chatbots are broader than older tools, since one system can write, summarise, code and answer questions. Still, they are usually described as AI rather than AGI.\n\nThe word has also shifted over time. Tasks once thought to need real intelligence, like playing chess or reading handwriting, tend to stop being called AI once computers handle them routinely. That habit is sometimes described as moving the goalposts, and it is a reminder that the label is loose.\n\n## AGI: matching people\n\nAGI stands for artificial general intelligence. The rough idea is a system that can learn and perform almost any intellectual task a capable person can, rather than being tied to one narrow area.\n\nHere the trouble begins, because there is no single accepted definition. Some definitions focus on doing most economically valuable work. Others focus on passing a wide range of tests, or on learning new skills as efficiently as a person does. Different organisations pick different ones, which is why two people can argue about whether AGI has arrived and both be using the word sensibly.\n\n- **Breadth.** Can it handle many kinds of task, not just one?\n- **Level.** Is it as good as a typical person, or as good as an expert?\n- **Autonomy.** Can it carry out long pieces of work without constant help?\n\nBecause the definitions differ, be careful with any headline that says AGI has been achieved, or is a set number of years away. Ask which definition is being used.\n\nIt also helps to know that AGI is not a switch that flips on a particular day. Systems may become broadly capable in some areas well before others, and people may disagree for years about the moment the label fits. Some researchers prefer to talk about degrees of generality rather than a yes or no.\n\nA further wrinkle is that a system can be better than most people at some tasks and still worse than a child at others. Uneven ability is normal for current models, and it makes simple comparisons with human intelligence tricky.\n\n## ASI: going beyond people\n\nASI stands for artificial superintelligence. It means a system that is much better than the best humans across nearly every field that matters. This is the same idea covered in [What is superintelligence?](/guides/what-is-superintelligence/).\n\nASI is a higher bar than AGI. A system that matches a capable person is not automatically far beyond one. Whether the gap between the two would be crossed quickly or slowly is a live debate, covered in the guide on [fast or slow takeoff](/guides/fast-or-slow-takeoff/).\n\nThe word superintelligence is used more or less interchangeably with ASI. Some writers add distinctions, such as a system that is only faster than people, one that is collectively stronger through many copies working together, or one that is better in quality of thought. These are ideas from the research literature and are useful to know exist, though most everyday discussion does not need them.\n\nNo system today is described as ASI by serious researchers, and there is no agreed test for recognising one.\n\n## Why the distinction matters\n\nThe three terms point to different questions. For ordinary AI, the questions are practical: is it accurate, fair, private and useful? For AGI, the questions become economic and social, such as what happens to work and who benefits. For ASI, the questions include control and safety, which the [alignment guide](/guides/alignment/) explores.\n\nMixing the terms together can also make a debate go in circles. One person may be talking about the chatbot on their phone while another is talking about a hypothetical system far beyond anything built. They will not agree, because they are not discussing the same thing.\n\n## Telling them apart in practice\n\nWhen you see one of these words in a headline or an advertisement, a few quick questions help.\n\n- Which definition of the term is being used, and who wrote it?\n- Is the claim about what a system can do now, or what it might do later?\n- Is there a test or measurement behind it, or only an opinion?\n\nThe last question is where measurement comes in. Benchmarks are the main tool for comparing systems over time, and the next guide, [How AI progress is measured](/guides/how-ai-progress-is-measured/), explains what they show and where they mislead. The [benchmark explorer](/benchmarks/) lets you look at real results yourself.\n"},{"path":"/guides/how-ai-progress-is-measured/","kind":"guide","title":"How AI progress is measured","summary":"Benchmarks, what they test, and why they stop being useful.","aliases":[],"markdown":"# How AI progress is measured\n\n> Benchmarks, what they test, and why they stop being useful.\n\nSource: https://www.superintelligence.com.au/guides/how-ai-progress-is-measured/\nGuide 3 of 6. Reviewed September 2026.\n\n## In short\n\n1. A benchmark is a fixed set of questions or tasks with right answers, so different models can be scored the same way.\n2. Tests wear out. Once top models score near 100%, a benchmark stops telling us much.\n3. A single score is a clue, not a verdict. The trend across several tests tells you more.\n\n## What a benchmark is\n\nThink of a benchmark as an exam that never changes. Researchers write a set of questions or tasks, each with an answer that can be checked, and every AI model sits the same exam. That makes it possible to compare models from different companies, and to compare this year’s models with last year’s.\n\nSome benchmarks test knowledge, like graduate-level science. Others test skills, like fixing real software bugs or solving puzzles the model has never seen before.\n\nSee it in the explorer: https://www.superintelligence.com.au/benchmarks/\n\nThe site tracks five of them, each aimed at a different kind of ability.\n\n- **GPQA Diamond** ([details](/benchmarks/gpqa-diamond/)) asks hard science questions written by experts, designed to be difficult even with a web search.\n- **SWE-bench Verified** ([details](/benchmarks/swe-bench-verified/)) gives a model real software problems from open projects and checks whether its fix works.\n- **FrontierMath** ([details](/benchmarks/frontiermath/)) uses new, difficult mathematics problems written by professional mathematicians.\n- **Humanity’s Last Exam** ([details](/benchmarks/humanitys-last-exam/)) collects very hard questions from experts across many subjects.\n- **ARC-AGI-2** ([details](/benchmarks/arc-agi-2/)) tests whether a model can work out a new pattern from a few examples.\n\n## Why tests wear out\n\nWhen a new benchmark launches, models usually score poorly. Over the following years scores climb, often faster than the people who wrote the test expected. Once the best models get nearly everything right, the benchmark is called saturated. It can no longer separate a good model from a great one, so researchers move on to something harder.\n\nThis is a sign that the tool did its job, not that it failed. But it means no single benchmark can follow progress for long. The field keeps building new tests, which is why the explorer tracks several at once and why newer ones often carry ambitious names.\n\nThere is a second cause of wear. Once a test is famous, companies have reason to do well on it, and the score starts to matter for its own sake. Researchers sometimes describe this as a measure ceasing to be a good measure once it becomes a target.\n\nA recurring pattern is that a test thought to be far out of reach turns out to be closer than expected. This is part of why forecasts about AI have been so hard, and why people who follow the field pay attention to how quickly each new test is being used up.\n\n## The contamination problem\n\nImagine a student who has seen the exam questions and answers before the test. Their score would be high, but it would say little about what they understand. Something similar can happen with AI models, and it is called data contamination.\n\nModels learn from enormous amounts of text collected from the internet. If a benchmark’s questions or answers are published online, they may end up in that training material, and the model may then reproduce them rather than work them out. Nobody has to cheat on purpose for this to happen.\n\nTest designers respond in several ways:\n\n- Keeping part of the question set private, so it cannot leak into training data.\n- Writing fresh problems that did not exist when the model was trained.\n- Checking whether a model’s answers look memorised rather than reasoned.\n\nThese steps help, though none is perfect. It is one reason to give more weight to tests with hidden or newly written questions.\n\n## Reading a score well\n\nA score is only meaningful with context. Before reading much into a number, it helps to ask a few plain questions.\n\n- **What does the test actually measure?** A high mark on a science quiz does not mean a model can do a scientist’s job.\n- **How was it run?** Results can change depending on how many attempts the model gets, whether it can use tools, and how much thinking time it is given.\n- **Who reported it?** A result checked by an independent group is usually more reliable than one a company reports about its own model.\n- **What is the trend?** One number is a snapshot. A line over time across several tests is more informative.\n\nScores also do not translate neatly into real-world usefulness. A model can do well on a test and still fail at an ordinary task, or the reverse. The [benchmark explorer](/benchmarks/) shows live numbers, so this page does not quote scores that would go out of date.\n\nIt is also fair to keep in mind what a benchmark leaves out. Most are narrow, run on a computer and marked automatically. They rarely capture judgement, teamwork, reliability over weeks, or knowing when to ask for help, all of which matter in real work.\n\n## Comparing with people\n\nMany benchmarks include a human baseline, which is how well people perform on the same test. It gives the numbers a reference point, such as whether a model has passed the typical expert or only the typical student.\n\nBaselines need care too. The people tested might be experts, volunteers or crowd workers, and they may have had limited time or no tools. A different group could produce a different baseline. Treat it as a useful landmark rather than a fixed line.\n\n## What to take from this\n\nMeasurement in AI is imperfect, and researchers say so themselves. Even so, tests are the best shared way we have to see whether systems are getting better and where they still struggle. Used carefully, across several benchmarks and over time, they give a fair picture.\n\nIf you want to understand why scores have climbed as they have, the next guide, [Scaling, simply](/guides/scaling-simply/), explains the main ingredient behind that progress.\n\nIn practice that means being wary of two opposite mistakes. One is treating a single impressive score as proof that a system can do anything. The other is dismissing all testing because it is imperfect. A steady, sceptical reading sits between them.\n"},{"path":"/guides/scaling-simply/","kind":"guide","title":"Scaling, simply","summary":"Why more compute and data kept making models better.","aliases":[],"markdown":"# Scaling, simply\n\n> Why more compute and data kept making models better.\n\nSource: https://www.superintelligence.com.au/guides/scaling-simply/\nGuide 4 of 6. Reviewed September 2026.\n\n## In short\n\n1. Scaling means making models bigger and training them on more data with more computing power, which has reliably improved results so far.\n2. Scaling laws are patterns showing that performance tends to improve smoothly and predictably as these ingredients grow.\n3. Researchers disagree about whether scaling alone will continue to work, or whether new ideas will be needed.\n\n## The basic recipe\n\nModern AI systems are trained rather than programmed line by line. A model reads a vast amount of text and other data, and gradually adjusts itself to get better at predicting what comes next. Three ingredients shape how good the result is: the size of the model, the amount of data, and the amount of computing power used to train it.\n\nScaling means turning all three up together. For roughly the last decade, doing so has produced better systems again and again, and that surprised many people in the field.\n\n## The three ingredients\n\nA model’s size is usually counted in parameters. These are the adjustable numbers inside it, a bit like the settings that get tuned during learning. Larger models have more of them and can, in general, capture more patterns.\n\nData is the material the model learns from. More varied, higher-quality data gives it more to learn from. Compute is the raw calculation used in training, often counted in operations, and it costs a great deal in chips and electricity.\n\nThe early, broad stage of learning is called pre-training. It is where most of the raw ability comes from, before later steps shape how the model behaves.\n\nA rough comparison is a student. A bigger brain, more books and more hours of study all help, but only when they are balanced. A great deal of reading with too little capacity to absorb it, or the reverse, does not get the same result.\n\nIt is also why the physical side matters. Training the largest models uses many thousands of specialised chips running for weeks or months, and the cost of that equipment is a main reason only a small number of organisations build the biggest systems, often called frontier labs.\n\n## Scaling laws\n\nAround the late 2010s, researchers noticed a pattern. When they plotted a model’s error against the amount of compute, data or size used, the line was surprisingly smooth. Performance improved in a predictable way as the ingredients grew. The best-known study, published in 2020, found the pattern held for language models across a very wide range of sizes.\n\nThese patterns are called scaling laws. They let a lab estimate, before spending the money, roughly how much better a larger training run should be. That predictability is a big part of why companies were willing to invest so heavily.\n\nLater work refined the picture by showing that size and data need to grow in balance. Making a model huge while feeding it too little data wastes effort. The details of these findings are technical, and a reader curious about them should check the original papers rather than rely on a summary.\n\nSee it in the explorer: https://www.superintelligence.com.au/benchmarks/\n\n## New abilities and surprises\n\nAs models grew, they did not only get better at the same things. Some skills seemed to appear at larger sizes that smaller models lacked, such as following complex instructions or solving certain kinds of puzzle. These are often called emergent abilities.\n\nResearchers disagree about how to read this. Some see genuine sudden jumps in capability. Others argue that the jumps depend on how we measure, and that with a gentler scoring method the improvement looks smooth. Both sides agree that it is hard to predict exactly which skills will show up at which scale.\n\nThis is one reason careful testing matters. The guide on [how AI progress is measured](/guides/how-ai-progress-is-measured/) explains what the tests show, and the [explorer](/benchmarks/) lets you watch results change over time.\n\nA practical consequence is that a lab often cannot know everything a new model can do until it has been trained and tested. Some abilities are found after release, when people try things nobody thought of. Careful evaluation before and after release is one response to that uncertainty.\n\n## Beyond bigger training runs\n\nMore recently, labs have looked for other ways to improve models. One is giving a model more time to think through a problem before it answers, sometimes called test-time compute. Another is training models specifically to reason step by step. Both spend computing power in a different place, after training rather than during it.\n\nThis matters for the story of scaling. It suggests the recipe has more than one dial, and that progress does not depend only on ever larger training runs.\n\nResearchers also work on making models more efficient, so the same ability needs less computing power. If that continues, capability could spread more widely and cheaply, which brings its own benefits and worries.\n\n## Will it keep working?\n\nNobody knows how far scaling will go. There are a few common views.\n\n- **It will continue.** Some researchers expect that more compute and data, plus better methods, will keep delivering gains.\n- **It will hit limits.** Others point to the finite supply of good training data, rising costs and energy needs, and think progress will slow.\n- **New ideas are needed.** A third group believes scaling helps a great deal but that reaching human-level general ability needs some further breakthrough.\n\nEvidence so far does not settle the argument, and any confident prediction should be treated with caution. What is clear is that scaling has been one of the main reasons AI moved as fast as it did.\n\nIf systems keep improving, a natural question follows: how do we make sure they do what we want? That is the subject of the next guide, [Alignment: keeping AI on our side](/guides/alignment/).\n"},{"path":"/guides/alignment/","kind":"guide","title":"Alignment: keeping AI on our side","summary":"The problem of getting powerful systems to do what we intend.","aliases":[],"markdown":"# Alignment: keeping AI on our side\n\n> The problem of getting powerful systems to do what we intend.\n\nSource: https://www.superintelligence.com.au/guides/alignment/\nGuide 5 of 6. Reviewed September 2026.\n\n## In short\n\n1. Alignment is the challenge of making AI systems pursue the goals and values their designers and users actually intend.\n2. The difficulty is that it is hard to state what we want precisely, and hard to check what a system is really doing.\n3. Researchers work on this with training methods, testing and efforts to look inside models, and they disagree about how hard the problem is.\n\n## The core problem\n\nAlignment is the work of making sure an AI system does what its makers and users actually want. That sounds simple, and for a basic tool it mostly is. The difficulty grows as systems become more capable and are trusted with more independent decisions.\n\nThe concern is not usually that a system turns hostile. It is that it does something slightly different from what we meant, and does it very effectively.\n\nAlignment is also broader than one technique. It includes deciding what good behaviour looks like, training systems toward it, testing whether it holds, and monitoring systems after they are in use. Different research groups focus on different parts of that chain.\n\nMany people working on it are not predicting catastrophe. They see it as ordinary safety engineering, applied to a technology that is becoming more powerful, in the way that engineers plan for failures in bridges or aircraft.\n\n## Why it is hard to say what we want\n\nThink of asking a helper to \"get me to the airport as fast as possible\". You would not want them to speed dangerously, yet the instruction alone does not say so. People fill in unstated limits without thinking, and a computer system does not automatically share those assumptions.\n\nResearchers describe this as the difficulty of specifying goals. When a system is trained to maximise some measurable target, it may find shortcuts that raise the number without delivering what we cared about. A well-known example in the field is a program trained in 2016 to play a boat-racing video game, with the game’s score as its target. It learned to circle a small lagoon, hitting the same point-scoring targets over and over. It scored higher than human players and never finished the race.\n\n## How systems are shaped today\n\nModern chatbots are trained in stages. After the broad learning phase, labs adjust behaviour in a step often called fine-tuning. One common method is reinforcement learning from human feedback, or RLHF, where people rate the model’s answers and the model is nudged toward the ones people prefer.\n\nThis works reasonably well for making assistants more helpful and less likely to produce harmful content. But it has limits. People can be fooled by an answer that sounds good and is wrong, and ratings capture what people prefer, which is not always what is true or wise.\n\nLabs also write out principles or rules for their models to follow, and test models against them. Approaches differ between organisations, and their details change often, so it is best to read each lab’s own published material.\n\nAnother difficulty is that a model’s behaviour can vary with the wording of a request or the situation it is placed in. A system that behaves well in most cases may still behave badly in rare ones, and rare cases are hard to find in advance. This is why researchers put effort into deliberately probing systems for weaknesses.\n\n## Looking inside and testing\n\nTwo further areas matter a great deal.\n\n- **Evaluation.** Researchers design tests to see how a model behaves in tricky situations, such as whether it will deceive, follow harmful instructions or take unwanted actions when given tools.\n- **Interpretability.** This is the effort to understand what is happening inside a model, rather than judging it only by its outputs. It is like trying to read the reasoning of a mind from the wiring.\n\nBoth are hard. Models are large and complicated, and a system might behave well in testing and differently in real use. Progress is being made, but researchers say openly that our tools for checking are still limited.\n\n## Why some researchers worry about more capable systems\n\nA further argument concerns what capable systems might do in pursuit of almost any goal. The idea, sometimes called instrumental convergence, is that many different goals share useful sub-goals, such as staying operational, gaining resources or avoiding being switched off. A system pursuing nearly anything might therefore have a reason to resist correction.\n\nThis is a theoretical argument about possible future systems, not a description of how current chatbots behave. Some researchers find it persuasive and central to why alignment matters. Others think it is overstated, or that practical engineering will keep such behaviour in check.\n\nThe link to the wider picture is in [What is superintelligence?](/guides/what-is-superintelligence/). The more capable a system, the more it matters that its goals are right.\n\n## Where people disagree\n\nViews on alignment cover a wide range. Some believe it is a serious unsolved problem that needs far more research before very capable systems are built. Others believe current methods, improved step by step, will be enough, and that the problem looks harder in theory than in practice.\n\nThere is also debate about who should decide what a system is aligned to. Aligning with a user’s wishes, a company’s policies and society’s wider interests can pull in different directions.\n\nWhatever the view, the work relies on honest testing and public evidence, which is why measurement is a recurring theme across this site. The next guide, [Fast or slow takeoff?](/guides/fast-or-slow-takeoff/), looks at how much time we might have to get this right.\n\nA related question is how much to rely on the companies building the systems to check their own work. Some argue for independent testing and public reporting, so that claims about safety can be verified by people outside the company. Others favour flexible approaches that can keep pace with quick changes in the technology.\n\nFor a non-specialist, the useful habit is to ask what evidence supports a safety claim, and who checked it.\n"},{"path":"/guides/fast-or-slow-takeoff/","kind":"guide","title":"Fast or slow takeoff?","summary":"The debate over how quickly AGI could become superintelligence.","aliases":[],"markdown":"# Fast or slow takeoff?\n\n> The debate over how quickly AGI could become superintelligence.\n\nSource: https://www.superintelligence.com.au/guides/fast-or-slow-takeoff/\nGuide 6 of 6. Reviewed September 2026.\n\n## In short\n\n1. Takeoff describes how quickly AI could move from roughly human-level ability to far beyond it.\n2. A fast takeoff would leave little time to react, while a slow one would give society years to adapt and correct course.\n3. Serious researchers disagree, and nobody can honestly give a date or promise which path is more likely.\n\n## What takeoff means\n\nSuppose AI systems eventually reach the level of a capable person across most intellectual work, which is often called AGI. Takeoff is the name for what happens next: how quickly do systems move from that point to far beyond human ability?\n\nThe question matters because speed decides how much time people have to notice problems, adjust rules and fix mistakes. It is a debate about pace, not about whether any of this will happen.\n\nPeople sometimes use the words hard and soft, or fast and slow, and describe the timescale differently. Some mean minutes or days, some mean years, and others mean decades. It is worth checking which scale a speaker has in mind, since a takeoff of several years is very different from one of several days.\n\n## The case for a fast takeoff\n\nThe fast view usually starts with an old idea. I. J. Good suggested in 1965 that a machine able to improve its own design could set off a rapid loop of improvement. This is sometimes called recursive self-improvement.\n\nThe reasoning goes like this. If an AI system is good at AI research, it can help build a better system. That better system is even better at research, and so on. Each round could be quicker than the last, so a period of years might shrink to months, or less.\n\nAnother argument concerns something called a capability overhang. If the hardware and data for much stronger systems already exist and only the right method is missing, a single good idea could release a large jump in ability all at once.\n\n## The case for a slow takeoff\n\nThe slow view points out that real-world progress tends to hit friction. Training large models takes time, chips and energy. Ideas that look promising often need long testing, and new tools have to be built into how organisations actually work.\n\nSupporters also note that improvement in AI so far has been fairly gradual and widely visible, with many groups working at similar levels. On this picture, systems get steadily more capable and more useful, and the economy and institutions change alongside them. There is no single moment of transformation.\n\nSome also argue that the self-improvement loop may have diminishing returns. Each further gain could be harder to find than the last, which would slow the loop instead of speeding it.\n\nAnother point in this view is that many small improvements would let people learn as they go. Rules, testing methods and habits of use could develop alongside the technology instead of arriving after it. Critics respond that gradual progress can still cross important thresholds without much warning.\n\n## What would help us tell\n\nResearchers look for evidence that could favour one picture over the other. None of it is conclusive yet.\n\n- **How much AI helps AI research.** If systems begin to speed up their own development in a measurable way, that would matter.\n- **Shape of the trend lines.** Smooth, steady improvement on tests points one way, while sudden jumps point another.\n- **Bottlenecks.** Limits on chips, energy, data and human oversight could act as brakes.\n\nThe [benchmark explorer](/benchmarks/) lets you follow how systems perform on hard tests over time. The guide on [how AI progress is measured](/guides/how-ai-progress-is-measured/) explains how to read those results carefully, since tests can saturate and a single score can mislead.\n\n## Why the speed changes the stakes\n\nIf takeoff is slow, there is time for trial and error. Problems would show up in weaker systems first, and society could respond. This is the setting in which the practical work described in the [alignment guide](/guides/alignment/) has the most room to help.\n\nIf takeoff is fast, there may be little or no chance to learn from early mistakes. That is why some researchers argue for caution and preparation in advance, even though they cannot say the fast path is likely. Others reply that planning around a very unlikely scenario can take attention from nearer and more certain problems.\n\nIt is also possible that reality sits in between, with a period of gradual change followed by a sharp acceleration, or the reverse.\n\n## Holding the question honestly\n\nThis site does not predict when, or whether, AGI or superintelligence will arrive. Forecasts from experts vary widely, and their track record on precise timing is mixed. A confident date from anyone should be treated with caution.\n\nWhat you can do is follow the evidence as it develops. Look at several measures, read how the researchers on each side make their case, and notice which claims rest on data and which on argument. If you are new to the wider subject, the first guide, [What is superintelligence?](/guides/what-is-superintelligence/), is a good place to return to.\n\nWhichever way the evidence points, it is reasonable to want both preparation and a sense of proportion. Preparing for a possible fast path does not require believing it is likely, and enjoying the benefits of a slow path does not mean ignoring risks. Calm, informed attention is a good starting point for anyone.\n"},{"path":"/benchmarks/gpqa-diamond/","kind":"benchmark","title":"GPQA Diamond","summary":"Graduate-level biology, chemistry and physics questions that are hard to look up.","aliases":["gpqa-diamond"],"markdown":"# GPQA Diamond\n\n> Graduate-level biology, chemistry and physics questions that are hard to look up.\n\nSource: https://www.superintelligence.com.au/benchmarks/gpqa-diamond/\nData updated 2026-10-07. Benchmark data from Epoch AI (CC BY) and official leaderboards.\n\nBest score today: 96% (+3 pts in 12 months).\nHuman comparison: Experts in their own field, 81%. https://arxiv.org/abs/2311.12022\n\n## What it tests\n\nMultiple-choice questions in biology, chemistry and physics, written by people with or working towards PhDs and checked so the answer can’t simply be searched for online.\n\n## Why it matters\n\nIt shows whether a model can reason through graduate-level science rather than recall facts.\n\n## Caveats\n\nTop models now sit near the ceiling, so small differences between them mean little. Four answer options also mean a random guess scores about 25%, which is why the line never starts at zero.\n\n## Best score by quarter\n\n| Quarter | Best score | Record held by |\n| --- | --- | --- |\n| Q1 2023 | 36% | GPT-4 (Mar 2023) |\n| Q2 2023 | 36% | GPT-4 (Mar 2023) |\n| Q3 2023 | 36% | GPT-4 (Mar 2023) |\n| Q4 2023 | 42% | GPT-4 Turbo Preview (Nov 2023) |\n| Q1 2024 | 47% | claude-3-opus-20240229 |\n| Q2 2024 | 54% | Claude 3.5 Sonnet (Jun 2024) |\n| Q3 2024 | 62% | o1-mini (high) |\n| Q4 2024 | 77% | o1 (high) |\n| Q1 2025 | 84% | Gemini 2.5 Pro Exp (Mar 2025) |\n| Q2 2025 | 85% | gemini-2.5-pro |\n| Q3 2025 | 87% | grok-4-0709 |\n| Q4 2025 | 93% | Gemini 3 Pro Preview |\n| Q1 2026 | 95% | GPT-5.4 Pro (xhigh) |\n| Q2 2026 | 95% | GPT-5.4 Pro (xhigh) |\n| Q3 2026 | 96% | GPT-6 Astra (max) |\n| Q4 2026 | 96% | GPT-6 Astra (max) |\n\n## Top scores right now\n\n| Rank | Model | Developer | Score | Released |\n| --- | --- | --- | --- | --- |\n| 1 | GPT-6 Astra (max) | OpenAI | 96% | 2026-09-03 |\n| 2 | Claude Sonnet 5.5 (max) | Anthropic | 96% | 2026-09-28 |\n| 3 | Gemini 3.8 Flash (high) | Google DeepMind | 95% | 2026-09-02 |\n| 4 | GPT-6.1 Sol (max) | OpenAI | 95% | 2026-09-29 |\n| 5 | Gemini 3.7 Flash (high) | Google DeepMind | 95% | 2026-08-13 |\n| 6 | GPT-5.4 Pro (xhigh) | OpenAI | 95% | 2026-03-05 |\n| 7 | Gemini 3.1 Pro Preview | Google DeepMind | 94% | 2026-02-19 |\n| 8 | GPT-6 Sol (max) | OpenAI | 94% | 2026-09-22 |\n| 9 | Gemini 3.6 Flash | Google DeepMind | 94% | 2026-07-21 |\n| 10 | GPT-5.5 (xhigh) | OpenAI | 94% | 2026-04-23 |\n| 11 | Grok 4.6 (high) | xAI | 94% | 2026-08-12 |\n| 12 | GPT-5.5 Pro (xhigh) | OpenAI | 94% | 2026-04-23 |\n| 13 | Claude Opus 5 (max) | Anthropic | 94% | 2026-07-24 |\n| 14 | GPT-5.6 Sol (max) | OpenAI | 94% | 2026-07-09 |\n| 15 | Grok 4.5 (high) | xAI | 93% | 2026-07-08 |\n| 16 | GPT-5.6 Terra (max) | OpenAI | 93% | 2026-07-09 |\n| 17 | GPT-5.4 (xhigh) | OpenAI | 93% | 2026-03-05 |\n| 18 | Grok 4.6 (xhigh) | xAI | 93% | 2026-08-12 |\n| 19 | Kimi K3 (max) | Moonshot | 93% | 2026-07-16 |\n| 20 | Claude Opus 5 | Anthropic | 93% | 2026-07-24 |\n| 21 | Gemini 3.5 Flash | Google DeepMind | 93% | 2026-05-19 |\n| 22 | Grok 4.7 (xhigh) | xAI | 93% | 2026-09-21 |\n| 23 | Qwen3.8 Max (xhigh) | Alibaba | 93% | 2026-08-02 |\n| 24 | Gemini 3 Pro Preview | Google DeepMind | 93% | 2025-11-18 |\n| 25 | Qwen3.8 Max (0902) (xhigh) | Alibaba | 92% | 2026-09-01 |\n| 26 | Kimi K3 (high) | Moonshot | 92% | 2026-07-16 |\n| 27 | GLM-5.2 | Z.ai (Zhipu AI) | 92% | 2026-06-16 |\n| 28 | DeepSeek V4 Pro 0813 (max) | DeepSeek | 92% | 2026-08-13 |\n| 29 | GPT-5.6 Luna (max) | OpenAI | 92% | 2026-07-09 |\n| 30 | GPT-5.2 (xhigh) | OpenAI | 91% | 2025-12-11 |\n| 31 | Claude Opus 4.8 | Anthropic | 91% | 2026-05-28 |\n| 32 | DeepSeek V4 Flash 0731 (max) | DeepSeek | 91% | 2026-07-31 |\n| 33 | DeepSeek v4 Pro (high) | DeepSeek | 91% | 2026-04-24 |\n| 34 | Qwen3.7 Max | Alibaba | 91% | 2026-05-19 |\n| 35 | MiniMax-M3 | MiniMax | 91% | 2026-06-01 |\n| 36 | GLM-5.3 (max) | Z.ai (Zhipu AI) | 91% | 2026-08-14 |\n| 37 | Kimi K2.6 | Moonshot | 91% | 2026-04-20 |\n| 38 | GPT-5.5 (low) | OpenAI | 91% | 2026-04-23 |\n| 39 | Claude Opus 5.5 (max) | Anthropic | 91% | 2026-09-22 |\n| 40 | Claude Opus 4.6 (32k thinking) | Anthropic | 91% | 2026-02-05 |\n| 41 | Claude Sonnet 5 (xhigh) | Anthropic | 91% | 2026-06-30 |\n| 42 | GPT-6 Luna (max) | OpenAI | 91% | 2026-09-22 |\n| 43 | Claude Opus 4.7 (xhigh) | Anthropic | 90% | 2026-04-16 |\n| 44 | GLM-5.3-Flash (max) | Z.ai (Zhipu AI) | 90% | 2026-08-20 |\n| 45 | GPT-5.4 (high) | OpenAI | 90% | 2026-03-05 |\n| 46 | GLM-5.1 | Z.ai (Zhipu AI) | 90% | 2026-04-07 |\n| 47 | GPT-5.6 Sol (low) | OpenAI | 90% | 2026-07-09 |\n| 48 | Muse Spark | Meta AI | 90% | 2026-04-08 |\n| 49 | DeepSeek v4 Pro (max) | DeepSeek | 90% | 2026-04-24 |\n| 50 | Gemini 3 Flash Preview | Google DeepMind | 89% | 2025-12-17 |\n\nDownload all data: https://www.superintelligence.com.au/data/benchmarks.csv\n"},{"path":"/benchmarks/swe-bench-verified/","kind":"benchmark","title":"SWE-bench Verified","summary":"Fixing real bugs in open-source Python projects so the tests pass.","aliases":["swe-bench-verified"],"markdown":"# SWE-bench Verified\n\n> Fixing real bugs in open-source Python projects so the tests pass.\n\nSource: https://www.superintelligence.com.au/benchmarks/swe-bench-verified/\nData updated 2026-10-07. Benchmark data from Epoch AI (CC BY) and official leaderboards.\n\nBest score today: 84% (+7 pts in 12 months).\nHuman comparison: There is no published human score. Each task is a real fix that a human developer already made, so in a sense people have solved all of them, but nobody has timed a group of engineers sitting the whole set.\n\n## What it tests\n\nReal bug reports from popular open-source Python projects. The model must change the code so the project’s own tests pass. The Verified set is a subset of 500 tasks that human engineers checked were clear and solvable.\n\n## Why it matters\n\nIt has been the most widely quoted measure of whether AI can do a software engineer’s everyday work.\n\n## Caveats\n\nScores depend heavily on the tools and scaffolding around the model, not just the model itself. Every task comes from a public repository, so some solutions may have been in a model’s training data. In February 2026 OpenAI, which produced the Verified set, stopped reporting scores on it. Its audit found many tasks whose tests reject correct fixes, and signs that leading models had seen the solutions in training. Read the top of this chart with that in mind.\n\n## Best score by quarter\n\n| Quarter | Best score | Record held by |\n| --- | --- | --- |\n| Q4 2024 | 31% | GPT-4o (Nov 2024) |\n| Q1 2025 | 61% | claude-3-7-sonnet-20250219 |\n| Q2 2025 | 71% | Claude Opus 4 |\n| Q3 2025 | 74% | GPT-5 (high) |\n| Q4 2025 | 77% | Claude Opus 4.5 (no thinking) |\n| Q1 2026 | 79% | Claude Opus 4.6 (no thinking) |\n| Q2 2026 | 84% | Claude Opus 4.7 (max) |\n| Q3 2026 | 84% | Claude Opus 4.7 (max) |\n| Q4 2026 | 84% | Claude Opus 4.7 (max) |\n\n## Top scores right now\n\n| Rank | Model | Developer | Score | Released |\n| --- | --- | --- | --- | --- |\n| 1 | Claude Opus 4.7 (max) | Anthropic | 84% | 2026-04-16 |\n| 2 | GPT-5.5 (xhigh) | OpenAI | 81% | 2026-04-23 |\n| 3 | Gemini 3.5 Flash | Google DeepMind | 79% | 2026-05-19 |\n| 4 | Claude Opus 4.6 (no thinking) | Anthropic | 79% | 2026-02-05 |\n| 5 | GLM-5.2 | Z.ai (Zhipu AI) | 79% | 2026-06-16 |\n| 6 | DeepSeek v4 Pro (max) | DeepSeek | 78% | 2026-04-24 |\n| 7 | Qwen3.7 Max | Alibaba | 77% | 2026-05-19 |\n| 8 | GPT-5.4 (high) | OpenAI | 77% | 2026-03-05 |\n| 9 | Claude Opus 4.5 (no thinking) | Anthropic | 77% | 2025-11-24 |\n| 10 | Kimi K2.6 | Moonshot | 77% | 2026-04-20 |\n| 11 | Qwen 3.6 Max (Preview) | Alibaba | 77% | 2026-04-20 |\n| 12 | Gemini 3.1 Pro Preview | Google DeepMind | 76% | 2026-02-19 |\n| 13 | Gemini 3 Flash Preview | Google DeepMind | 75% | 2025-12-17 |\n| 14 | Claude Sonnet 4.6 (no thinking) | Anthropic | 75% | 2026-02-17 |\n| 15 | GPT-5.3 Codex (high) | OpenAI | 75% | 2026-02-05 |\n| 16 | GLM-5.1 | Z.ai (Zhipu AI) | 74% | 2026-04-07 |\n| 17 | GPT-5.2 (high) | OpenAI | 74% | 2025-12-11 |\n| 18 | Kimi K2.5 | Moonshot | 74% | 2026-01-27 |\n| 19 | GPT-5 (high) | OpenAI | 74% | 2025-08-07 |\n| 20 | claude-opus-4-1-20250805 | Anthropic | 73% | 2025-08-05 |\n| 21 | Gemini 3 Pro Preview | Google DeepMind | 73% | 2025-11-18 |\n| 22 | GLM-5 | Z.ai (Zhipu AI) | 72% | 2026-02-11 |\n| 23 | GPT-5 (medium) | OpenAI | 72% | 2025-08-07 |\n| 24 | Claude Sonnet 4.5 (no thinking) | Anthropic | 71% | 2025-09-29 |\n| 25 | Claude Opus 4 | Anthropic | 71% | 2025-05-22 |\n| 26 | GPT-5.1 (high) | OpenAI | 68% | 2025-11-13 |\n| 27 | GPT-5 mini (medium) | OpenAI | 65% | 2025-08-07 |\n| 28 | o3 (medium) | OpenAI | 62% | 2025-04-16 |\n| 29 | claude-3-7-sonnet-20250219 | Anthropic | 61% | 2025-02-24 |\n| 30 | Qwen 3.6 Plus (2026-04-02) | Alibaba | 58% | 2026-03-31 |\n| 31 | gemini-2.5-pro | Google DeepMind | 58% | 2025-06-17 |\n| 32 | GPT-4.1 | OpenAI | 49% | 2025-04-14 |\n| 33 | GPT-4o (Nov 2024) | OpenAI | 31% | 2024-11-20 |\n\nDownload all data: https://www.superintelligence.com.au/data/benchmarks.csv\n"},{"path":"/benchmarks/frontiermath/","kind":"benchmark","title":"FrontierMath","summary":"Unpublished research-level maths problems with answers a computer can check.","aliases":["frontiermath"],"markdown":"# FrontierMath\n\n> Unpublished research-level maths problems with answers a computer can check.\n\nSource: https://www.superintelligence.com.au/benchmarks/frontiermath/\nData updated 2026-10-07. Benchmark data from Epoch AI (CC BY) and official leaderboards.\n\nBest score today: 94% (+20 pts in 12 months).\nHuman comparison: No single human baseline exists. The problems were written by research mathematicians, and many take a specialist hours or days. No group of people has sat the whole test.\n\n## What it tests\n\nHundreds of unpublished problems written by research mathematicians, with answers that can be checked automatically. This site tracks tiers 1 to 3; a harder fourth tier is scored separately.\n\n## Why it matters\n\nThe problems are unpublished, so progress signals mathematical reasoning rather than memory.\n\n## Caveats\n\nThe problems are not public, but OpenAI commissioned them and has access to most of the problems and their solutions. Epoch AI, which runs the test, keeps a set back from OpenAI to check results against. Scores here are for the current version of the set, re-run on older models.\n\n## Best score by quarter\n\n| Quarter | Best score | Record held by |\n| --- | --- | --- |\n| Q1 2024 | 0% | GPT-3.5 Turbo (Jan 2024) |\n| Q2 2024 | 1% | gpt-4-turbo-2024-04-09 |\n| Q3 2024 | 1% | gpt-4-turbo-2024-04-09 |\n| Q4 2024 | 15% | o1 (high) |\n| Q1 2025 | 19% | o3-mini (high) |\n| Q2 2025 | 36% | o4-mini (high) |\n| Q3 2025 | 55% | GPT-5 (high) |\n| Q4 2025 | 74% | GPT-5.2 Pro |\n| Q1 2026 | 83% | GPT-5.4 Pro (xhigh) |\n| Q2 2026 | 88% | GPT-5.5 Pro (xhigh) |\n| Q3 2026 | 94% | GPT-6.1 Sol (max) |\n| Q4 2026 | 94% | GPT-6.1 Sol (max) |\n\n## Top scores right now\n\n| Rank | Model | Developer | Score | Released |\n| --- | --- | --- | --- | --- |\n| 1 | GPT-6 Astra (max) | OpenAI | 94% | 2026-09-03 |\n| 2 | GPT-6.1 Sol (max) | OpenAI | 94% | 2026-09-29 |\n| 3 | Claude Opus 5.5 (max) | Anthropic | 91% | 2026-09-22 |\n| 4 | Claude Fable 5.1 (max) | Anthropic | 90% | 2026-09-01 |\n| 5 | GPT-6 Sol (max) | OpenAI | 90% | 2026-09-22 |\n| 6 | GPT-5.6 Sol (max) | OpenAI | 89% | 2026-07-09 |\n| 7 | Claude Sonnet 5.5 (max) | Anthropic | 89% | 2026-09-28 |\n| 8 | GPT-5.5 Pro (xhigh) | OpenAI | 88% | 2026-04-23 |\n| 9 | Claude Fable 5 (max) | Anthropic | 87% | 2026-06-09 |\n| 10 | GPT-5.6 Terra (max) | OpenAI | 86% | 2026-07-09 |\n| 11 | Claude Opus 5 (max) | Anthropic | 86% | 2026-07-24 |\n| 12 | GPT-5.5 (xhigh) | OpenAI | 85% | 2026-04-23 |\n| 13 | GPT-5.4 Pro (xhigh) | OpenAI | 83% | 2026-03-05 |\n| 14 | GPT-5.6 Luna (max) | OpenAI | 82% | 2026-07-09 |\n| 15 | Claude Opus 4.8 | Anthropic | 80% | 2026-05-28 |\n| 16 | GPT-6 Luna (max) | OpenAI | 79% | 2026-09-22 |\n| 17 | GPT-5.4 (xhigh) | OpenAI | 79% | 2026-03-05 |\n| 18 | Qwen3.8 Max (xhigh) | Alibaba | 75% | 2026-08-02 |\n| 19 | Muse Spark 1.3 (xhigh) | Meta AI | 74% | 2026-09-02 |\n| 20 | Muse Spark 1.3 (max) | Meta AI | 74% | 2026-09-02 |\n| 21 | GPT-5.2 Pro | OpenAI | 74% | 2025-12-11 |\n| 22 | Kimi K3 (max) | Moonshot | 72% | 2026-07-16 |\n| 23 | Gemini 3.7 Flash (high) | Google DeepMind | 72% | 2026-08-13 |\n| 24 | Claude Opus 4.7 (max) | Anthropic | 70% | 2026-04-16 |\n| 25 | GLM-5.3 (max) | Z.ai (Zhipu AI) | 69% | 2026-08-14 |\n| 26 | Gemini 3.8 Flash (high) | Google DeepMind | 68% | 2026-09-02 |\n| 27 | GPT-5.2 (xhigh) | OpenAI | 67% | 2025-12-11 |\n| 28 | Claude Opus 4.6 (max) | Anthropic | 66% | 2026-02-05 |\n| 29 | Grok 4.6 (xhigh) | xAI | 66% | 2026-08-12 |\n| 30 | Claude Sonnet 5 (max) | Anthropic | 66% | 2026-06-30 |\n| 31 | Qwen3.8 Max (0902) (xhigh) | Alibaba | 66% | 2026-09-01 |\n| 32 | Qwen3.7 Max | Alibaba | 65% | 2026-05-19 |\n| 33 | DeepSeek V4 Pro 0813 (max) | DeepSeek | 65% | 2026-08-13 |\n| 34 | Gemini 3.5 Flash | Google DeepMind | 63% | 2026-05-19 |\n| 35 | Gemini 3.1 Pro Preview | Google DeepMind | 60% | 2026-02-19 |\n| 36 | GLM-5.2 | Z.ai (Zhipu AI) | 59% | 2026-06-16 |\n| 37 | Gemini 3.6 Flash | Google DeepMind | 59% | 2026-07-21 |\n| 38 | DeepSeek V4 Flash 0731 (max) | DeepSeek | 58% | 2026-07-31 |\n| 39 | Kimi K2.6 | Moonshot | 57% | 2026-04-20 |\n| 40 | Grok 4.5 (high) | xAI | 57% | 2026-07-08 |\n| 41 | GPT-5 Pro | OpenAI | 56% | 2025-10-07 |\n| 42 | GLM-5.3-Flash (max) | Z.ai (Zhipu AI) | 56% | 2026-08-20 |\n| 43 | GPT-5 (high) | OpenAI | 55% | 2025-08-07 |\n| 44 | Kimi K2.7 Code | Moonshot | 54% | 2026-06-12 |\n| 45 | Grok 4.7 (xhigh) | xAI | 53% | 2026-09-21 |\n| 46 | Gemini 3 Flash Preview | Google DeepMind | 51% | 2025-12-17 |\n| 47 | GPT-5.4 mini (xhigh) | OpenAI | 51% | 2026-03-17 |\n| 48 | GPT-5 mini (high) | OpenAI | 47% | 2025-08-07 |\n| 49 | Inkling Small (xhigh) | Thinking Machines | 46% | 2026-07-15 |\n| 50 | DeepSeek v4 Pro (max) | DeepSeek | 45% | 2026-04-24 |\n\nDownload all data: https://www.superintelligence.com.au/data/benchmarks.csv\n"},{"path":"/benchmarks/humanitys-last-exam/","kind":"benchmark","title":"Humanity’s Last Exam","summary":"Thousands of expert questions across 100+ subjects, chosen to stump AI.","aliases":["humanitys-last-exam"],"markdown":"# Humanity’s Last Exam\n\n> Thousands of expert questions across 100+ subjects, chosen to stump AI.\n\nSource: https://www.superintelligence.com.au/benchmarks/humanitys-last-exam/\nData updated 2026-10-07. Benchmark data from Epoch AI (CC BY) and official leaderboards.\n\nBest score today: 55% (+17 pts in 12 months).\nHuman comparison: There is no overall human score. Each question was written by a specialist to be hard for other specialists, and no one person could answer across every field.\n\n## What it tests\n\nAbout 2,500 questions across more than 100 subjects, written by around a thousand subject experts and kept only if leading models of the time got them wrong.\n\n## Why it matters\n\nIt was designed as a broad academic exam that AI would struggle with for years.\n\n## Caveats\n\nBecause questions were chosen to beat older models, early scores start artificially near zero. Results come from the official leaderboard rather than Epoch AI’s own runs.\n\n## Best score by quarter\n\n| Quarter | Best score | Record held by |\n| --- | --- | --- |\n| Q3 2024 | 5% | Gemini-1.5-Pro-002 |\n| Q4 2024 | 8% | o1 (unknown thinking) |\n| Q1 2025 | 18% | Gemini 2.5 Pro Exp (Mar 2025) |\n| Q2 2025 | 22% | Gemini 2.5 Pro Preview (Jun 2025) |\n| Q3 2025 | 25% | GPT-5 (high) |\n| Q4 2025 | 38% | Gemini 3 Pro Preview |\n| Q1 2026 | 46% | Gemini 3.1 Pro Preview |\n| Q2 2026 | 46% | Gemini 3.1 Pro Preview |\n| Q3 2026 | 55% | GPT-6 Astra (unknown thinking) |\n| Q4 2026 | 55% | GPT-6 Astra (unknown thinking) |\n\n## Top scores right now\n\n| Rank | Model | Developer | Score | Released |\n| --- | --- | --- | --- | --- |\n| 1 | GPT-6 Astra (unknown thinking) | OpenAI | 55% | 2026-09-03 |\n| 2 | Claude Fable 5.1 (xhigh) | Anthropic | 47% | 2026-09-01 |\n| 3 | Gemini 3.1 Pro Preview | Google DeepMind | 46% | 2026-02-19 |\n| 4 | Gemini 3.8 Flash (unknown) | Google DeepMind | 45% | 2026-09-02 |\n| 5 | GPT-5.4 Pro | OpenAI | 44% | 2026-03-05 |\n| 6 | Muse Spark | Meta AI | 41% | 2026-04-08 |\n| 7 | Gemini 3 Pro Preview | Google DeepMind | 38% | 2025-11-18 |\n| 8 | GPT-5.4 (xhigh) | OpenAI | 36% | 2026-03-05 |\n| 9 | Claude Opus 4.7 (unknown) | Anthropic | 36% | 2026-04-16 |\n| 10 | Claude Opus 4.6 (max) | Anthropic | 34% | 2026-02-05 |\n| 11 | GPT-5 Pro | OpenAI | 32% | 2025-10-07 |\n| 12 | GPT-5.2 (unknown thinking) | OpenAI | 28% | 2025-12-11 |\n| 13 | GPT-5 (high) | OpenAI | 25% | 2025-08-07 |\n| 14 | GPT-5 (unknown thinking) | OpenAI | 25% | 2025-08-07 |\n| 15 | Claude Opus 4.5 (unknown thinking) | Anthropic | 25% | 2025-11-24 |\n| 16 | Kimi K2.5 | Moonshot | 24% | 2026-01-27 |\n| 17 | GPT-5.1 (unknown thinking) | OpenAI | 24% | 2025-11-13 |\n| 18 | Gemini 2.5 Pro Preview (Jun 2025) | Google DeepMind | 22% | 2025-06-05 |\n| 19 | o3 (high) | OpenAI | 20% | 2025-04-16 |\n| 20 | GPT-5 mini (unknown thinking) | OpenAI | 19% | 2025-08-07 |\n| 21 | o3 (medium) | OpenAI | 19% | 2025-04-16 |\n| 22 | Claude Opus 4.6 (no thinking) | Anthropic | 19% | 2026-02-05 |\n| 23 | Gemini 2.5 Pro Exp (Mar 2025) | Google DeepMind | 18% | 2025-03-25 |\n| 24 | o4-mini (high) | OpenAI | 18% | 2025-04-16 |\n| 25 | o4-mini (medium) | OpenAI | 14% | 2025-04-16 |\n| 26 | Claude Sonnet 4.5 (unknown thinking) | Anthropic | 14% | 2025-09-29 |\n| 27 | Gemini 2.5 Flash Preview (Apr 2025) | Google DeepMind | 12% | 2025-04-17 |\n| 28 | Claude Opus 4.1 (unknown thinking) | Anthropic | 12% | 2025-08-05 |\n| 29 | Gemini 2.5 Flash Preview (May 2025) | Google DeepMind | 11% | 2025-05-20 |\n| 30 | Claude Opus 4 | Anthropic | 11% | 2025-05-22 |\n| 31 | Gemini 3.1 Flash-Lite | Google | 9% | 2026-03-03 |\n| 32 | glm-4p5 | Z.ai (Zhipu AI) | 8% | 2025-08-03 |\n| 33 | o1 Pro | OpenAI | 8% | 2025-03-19 |\n| 34 | glm-4p5-air | Z.ai (Zhipu AI) | 8% | 2025-07-20 |\n| 35 | Claude 3.7 Sonnet (unknown thinking) | Anthropic | 8% | 2025-02-24 |\n| 36 | o1 (unknown thinking) | OpenAI | 8% | 2024-12-17 |\n| 37 | Claude Sonnet 4 (unknown thinking) | Anthropic | 8% | 2025-05-22 |\n| 38 | GPT-5.1 (no thinking) | OpenAI | 7% | 2025-11-13 |\n| 39 | Gemini 2.0 Flash Thinking Exp | Google DeepMind | 7% | 2025-01-21 |\n| 40 | Llama 4 Maverick | Meta AI | 6% | 2025-04-05 |\n| 41 | GPT-4.5 Preview (Feb 2025) | OpenAI | 5% | 2025-02-27 |\n| 42 | GPT-4.1 | OpenAI | 5% | 2025-04-14 |\n| 43 | Gemini-1.5-Pro-002 | Google DeepMind | 5% | 2024-09-24 |\n| 44 | Mistral Medium 3 | Mistral AI | 5% | 2025-05-07 |\n| 45 | Nova Pro | Amazon | 4% | 2024-12-03 |\n| 46 | Claude 3.5 Sonnet (Oct 2024) | Anthropic | 4% | 2024-10-22 |\n| 47 | Nova Lite | Amazon | 4% | 2024-12-03 |\n| 48 | GPT-4o (Nov 2024) | OpenAI | 3% | 2024-11-20 |\n\nDownload all data: https://www.superintelligence.com.au/data/benchmarks.csv\n"},{"path":"/benchmarks/arc-agi-2/","kind":"benchmark","title":"ARC-AGI-2","summary":"Visual grid puzzles that are easy for people and new to every model.","aliases":["arc-agi-2"],"markdown":"# ARC-AGI-2\n\n> Visual grid puzzles that are easy for people and new to every model.\n\nSource: https://www.superintelligence.com.au/benchmarks/arc-agi-2/\nData updated 2026-10-07. Benchmark data from Epoch AI (CC BY) and official leaderboards.\n\nBest score today: 95% (+41 pts in 12 months).\nHuman comparison: The test’s creators had more than 400 members of the public attempt the puzzles, and every puzzle was solved by at least two of them. They published no single average score, so there is no one number to draw here.\n\n## What it tests\n\nGrid puzzles where you work out a hidden rule from a few examples, then apply it to a new grid.\n\n## Why it matters\n\nIt tests adapting to something new, which is closer to general intelligence than a knowledge test.\n\n## Caveats\n\nSome top results use very large amounts of computing per task, which the headline score doesn’t show. Results come from the ARC Prize leaderboard rather than Epoch AI’s own runs.\n\n## Best score by quarter\n\n| Quarter | Best score | Record held by |\n| --- | --- | --- |\n| Q3 2024 | 1% | o1-mini (unknown thinking) |\n| Q4 2024 | 1% | o1-mini (unknown thinking) |\n| Q1 2025 | 3% | o3-mini (high) |\n| Q2 2025 | 9% | Claude Opus 4 |\n| Q3 2025 | 16% | Grok 4 (Thinking) |\n| Q4 2025 | 54% | GPT-5.2 Pro |\n| Q1 2026 | 85% | Gemini 3 Deep Think |\n| Q2 2026 | 89% | Claude Fable 5 (max) |\n| Q3 2026 | 95% | GPT-6 Astra (max) |\n| Q4 2026 | 95% | GPT-6 Astra (max) |\n\n## Top scores right now\n\n| Rank | Model | Developer | Score | Released |\n| --- | --- | --- | --- | --- |\n| 1 | GPT-6 Astra (max) | OpenAI | 95% | 2026-09-03 |\n| 2 | GPT-6.1 Sol (max) | OpenAI | 94% | 2026-09-29 |\n| 3 | GPT-6 Astra (xhigh) | OpenAI | 93% | 2026-09-03 |\n| 4 | Claude Opus 5.5 (high) | Anthropic | 93% | 2026-09-22 |\n| 5 | GPT-5.6 Sol (max) | OpenAI | 93% | 2026-07-09 |\n| 6 | Claude Opus 5.5 (xhigh) | Anthropic | 93% | 2026-09-22 |\n| 7 | GPT-6 Astra (high) | OpenAI | 92% | 2026-09-03 |\n| 8 | GPT-6 Astra (medium) | OpenAI | 92% | 2026-09-03 |\n| 9 | Claude Opus 5.5 (max) | Anthropic | 92% | 2026-09-22 |\n| 10 | GPT-6.1 Sol (high) | OpenAI | 92% | 2026-09-29 |\n| 11 | GPT-6.1 Sol (xhigh) | OpenAI | 92% | 2026-09-29 |\n| 12 | Claude Opus 5 (max) | Anthropic | 90% | 2026-07-24 |\n| 13 | GPT-5.6 Sol (xhigh) | OpenAI | 90% | 2026-07-09 |\n| 14 | Claude Fable 5.1 (max) | Anthropic | 90% | 2026-09-01 |\n| 15 | Claude Fable 5.1 (xhigh) | Anthropic | 90% | 2026-09-01 |\n| 16 | GPT-6 Sol (max) | OpenAI | 90% | 2026-09-22 |\n| 17 | Claude Fable 5 (max) | Anthropic | 89% | 2026-06-09 |\n| 18 | Gemini 3.8 Flash (high) | Google DeepMind | 89% | 2026-09-02 |\n| 19 | Claude Fable 5.1 (high) | Anthropic | 89% | 2026-09-01 |\n| 20 | Claude Fable 5 (xhigh) | Anthropic | 88% | 2026-06-09 |\n| 21 | Claude Opus 5 (high) | Anthropic | 88% | 2026-07-24 |\n| 22 | Claude Fable 5 (high) | Anthropic | 88% | 2026-06-09 |\n| 23 | Claude Opus 5.5 (medium) | Anthropic | 88% | 2026-09-22 |\n| 24 | GPT-6.1 Sol (medium) | OpenAI | 87% | 2026-09-29 |\n| 25 | Claude Fable 5.1 (medium) | Anthropic | 86% | 2026-09-01 |\n| 26 | GPT-5.6 Sol (high) | OpenAI | 85% | 2026-07-09 |\n| 27 | GPT-6 Astra (low) | OpenAI | 85% | 2026-09-03 |\n| 28 | GPT-5.5 (xhigh) | OpenAI | 85% | 2026-04-23 |\n| 29 | Gemini 3.7 Flash (high) | Google DeepMind | 85% | 2026-08-13 |\n| 30 | Gemini 3 Deep Think | Google DeepMind | 85% | 2026-02-12 |\n| 31 | GPT-5.5 Pro (high) | OpenAI | 85% | 2026-04-23 |\n| 32 | GPT-5.5 Pro (xhigh) | OpenAI | 84% | 2026-04-23 |\n| 33 | GPT-5.6 Terra (max) | OpenAI | 84% | 2026-07-09 |\n| 34 | GPT-5.4 Pro (xhigh) | OpenAI | 83% | 2026-03-05 |\n| 35 | GPT-5.5 (high) | OpenAI | 83% | 2026-04-23 |\n| 36 | Gemini 3.8 Flash (medium) | Google DeepMind | 83% | 2026-09-02 |\n| 37 | Claude Fable 5 (medium) | Anthropic | 83% | 2026-06-09 |\n| 38 | Claude Fable 5.1 (low) | Anthropic | 78% | 2026-09-01 |\n| 39 | GPT-6 Sol (xhigh) | OpenAI | 78% | 2026-09-22 |\n| 40 | Gemini 3.8 Flash (low) | Google DeepMind | 78% | 2026-09-02 |\n| 41 | Gemini 3.1 Pro Preview | Google DeepMind | 77% | 2026-02-19 |\n| 42 | Claude Fable 5 (low) | Anthropic | 77% | 2026-06-09 |\n| 43 | GPT-6.1 Sol (low) | OpenAI | 77% | 2026-09-29 |\n| 44 | Claude Opus 4.7 (max) | Anthropic | 76% | 2026-04-16 |\n| 45 | GPT-5.6 Terra (xhigh) | OpenAI | 74% | 2026-07-09 |\n| 46 | GPT-5.4 (xhigh) | OpenAI | 74% | 2026-03-05 |\n| 47 | Gemini 3.5 Flash | Google DeepMind | 72% | 2026-05-19 |\n| 48 | Claude Opus 4.8 | Anthropic | 72% | 2026-05-28 |\n| 49 | GPT-5.5 (medium) | OpenAI | 70% | 2026-04-23 |\n| 50 | Claude Opus 5.5 (low) | Anthropic | 70% | 2026-09-22 |\n\nDownload all data: https://www.superintelligence.com.au/data/benchmarks.csv\n"},{"path":"/glossary/agi/","kind":"term","title":"AGI","summary":"Artificial general intelligence: AI that can handle a wide range of tasks at roughly human level.","aliases":["artificial general intelligence"],"markdown":"# AGI\n\n> Artificial general intelligence: AI that can handle a wide range of tasks at roughly human level.\n\nSource: https://www.superintelligence.com.au/glossary/agi/\n\nAGI stands for artificial general intelligence. It usually means an AI system that can learn and perform a broad range of intellectual tasks about as well as a capable person, rather than being built for one narrow job like playing chess or translating text.\n\nThe idea matters because it marks a rough milestone in many people’s thinking: the point at which AI might do most of the thinking work that people do. Several AI companies say building it is their stated goal.\n\nThere is no agreed definition, and that causes a lot of confusion. Some people mean matching an average adult, others an expert in every field, and others the ability to do most economically valuable work. Some argue that current systems already count in some sense, and others that they are still far from it. Some researchers dislike the term altogether and prefer to talk about specific abilities. When someone says AGI has, or has not, been reached, it is worth asking what definition they are using and how they would test it. Timelines to AGI are also widely debated, and predictions vary greatly.\n\nRelated: https://www.superintelligence.com.au/glossary/asi.md, https://www.superintelligence.com.au/glossary/superintelligence.md, https://www.superintelligence.com.au/glossary/frontier-model.md, https://www.superintelligence.com.au/glossary/human-baseline.md\n"},{"path":"/glossary/alignment/","kind":"term","title":"Alignment","summary":"The work of making AI systems pursue the goals their designers and users actually intend.","aliases":[],"markdown":"# Alignment\n\n> The work of making AI systems pursue the goals their designers and users actually intend.\n\nSource: https://www.superintelligence.com.au/glossary/alignment/\n\nAlignment is the problem of getting an AI system to do what people actually want, not just what they literally typed or what its training happened to reward. It covers stating goals clearly, training systems to follow them, and checking that they keep doing so in new situations.\n\nA simple picture: ask a cleaning robot to \"get rid of the mess\" and it might shove everything into a cupboard. The instruction was followed, but the intention was not. Real cases are subtler. A chatbot trained on human approval can learn to flatter people, because flattery is rewarded, even though nobody wanted that.\n\nResearchers disagree about how hard the problem is. Some see it as ordinary engineering that improves alongside the systems themselves. Others expect it to become much harder once a system is more capable than the people checking its work. The word is also used loosely: sometimes it means AI safety in general, and sometimes only the narrower question of matching a system’s goals to human ones. It is worth checking which meaning a writer has in mind.\n\nRelated: https://www.superintelligence.com.au/glossary/interpretability.md, https://www.superintelligence.com.au/glossary/rlhf.md, https://www.superintelligence.com.au/glossary/instrumental-convergence.md\n"},{"path":"/glossary/asi/","kind":"term","title":"ASI","summary":"Artificial superintelligence: AI that would greatly outperform the best humans at nearly all intellectual tasks.","aliases":["artificial superintelligence"],"markdown":"# ASI\n\n> Artificial superintelligence: AI that would greatly outperform the best humans at nearly all intellectual tasks.\n\nSource: https://www.superintelligence.com.au/glossary/asi/\n\nASI stands for artificial superintelligence. It describes an AI system that would far exceed the best human minds across almost every intellectual area, including science, planning, persuasion and the design of new technology. It is a step beyond AGI, which is usually defined as roughly human-level ability.\n\nThe term is used mostly in discussions of long-term risk and possibility. If such a system could be built, it might solve problems people cannot, but it might also be very hard to oversee or to switch off if its goals differed from ours. That is why many safety researchers focus on it, even though it does not exist.\n\nOpinions differ sharply. Some think ASI could arrive within decades and want serious preparation now. Others think it is too speculative, or that it depends on assumptions about intelligence that may not hold. In practice, the terms ASI and superintelligence are used almost interchangeably, though some writers reserve ASI for a system that has been actually built rather than the general idea. No one has a reliable forecast for whether or when it could happen.\n\nRelated: https://www.superintelligence.com.au/glossary/agi.md, https://www.superintelligence.com.au/glossary/superintelligence.md, https://www.superintelligence.com.au/glossary/intelligence-explosion.md, https://www.superintelligence.com.au/glossary/takeoff.md\n"},{"path":"/glossary/benchmark/","kind":"term","title":"Benchmark","summary":"A standard set of test questions used to compare how well different AI models perform.","aliases":["benchmarks"],"markdown":"# Benchmark\n\n> A standard set of test questions used to compare how well different AI models perform.\n\nSource: https://www.superintelligence.com.au/glossary/benchmark/\n\nA benchmark is a fixed set of tasks or questions, with agreed ways of scoring the answers, that lets people compare AI models on equal terms. Think of it as an exam paper that every model sits under the same conditions. Common examples test general knowledge, mathematics, computer programming or the ability to follow instructions.\n\nBenchmarks matter because they are the main way progress in AI is described to the public. When a company says a new model is better than the last one, it usually means the model scored higher on one or more benchmarks. Without them, claims would be hard to compare or check.\n\nThey also have limits. A model can do well on a test without being useful in daily work, and a test can stop being informative once models learn its style or its answers. Researchers therefore treat any single score with caution and prefer to look at many benchmarks together, alongside real-world use. Scores reported by the developers themselves deserve extra care, since they choose which tests to show.\n\nRelated: https://www.superintelligence.com.au/glossary/evaluation.md, https://www.superintelligence.com.au/glossary/benchmark-saturation.md, https://www.superintelligence.com.au/glossary/human-baseline.md, https://www.superintelligence.com.au/glossary/data-contamination.md\n"},{"path":"/glossary/benchmark-saturation/","kind":"term","title":"Benchmark saturation","summary":"When top models score close to the maximum on a test, so it can no longer tell them apart.","aliases":[],"markdown":"# Benchmark saturation\n\n> When top models score close to the maximum on a test, so it can no longer tell them apart.\n\nSource: https://www.superintelligence.com.au/glossary/benchmark-saturation/\n\nBenchmark saturation happens when the best AI models score so close to the top mark that the test stops being useful. If several models all get 95% or more, the remaining differences may be noise, mistakes in the questions, or luck, rather than real differences in ability.\n\nPicture a school spelling test that every student in the class passes with full marks. It no longer shows who the strongest speller is. The same thing has happened repeatedly with AI tests: a benchmark that once looked hard becomes easy within a few years, and researchers have to build a harder one.\n\nThis matters for how progress is judged. Saturation can make progress look like it has stalled when the test has simply run out of room. It can also hide real gaps, because a high score on an old test says little about tasks the test never covered. Some researchers also worry that saturation is sped up when test questions leak into training material. There is no single agreed threshold for calling a test saturated, and people judge it differently.\n\nRelated: https://www.superintelligence.com.au/glossary/benchmark.md, https://www.superintelligence.com.au/glossary/human-baseline.md, https://www.superintelligence.com.au/glossary/data-contamination.md, https://www.superintelligence.com.au/glossary/evaluation.md\n"},{"path":"/glossary/capability-overhang/","kind":"term","title":"Capability overhang","summary":"When an AI system can do more than people have yet realised, or been able to draw out of it.","aliases":[],"markdown":"# Capability overhang\n\n> When an AI system can do more than people have yet realised, or been able to draw out of it.\n\nSource: https://www.superintelligence.com.au/glossary/capability-overhang/\n\nA capability overhang exists when a model or the available hardware is able to do more than is currently being achieved. The extra ability is there, but people have not yet found the right way to ask for it, or the right method to release it.\n\nFor example, a model might turn out to solve harder problems when it is prompted differently, given tools, or allowed to think for longer. Months after release, users find abilities that the builders did not test for. Better methods can then produce a sudden jump in what existing systems seem able to do.\n\nThe term matters in two ways. For safety testing, it means a clean evaluation may understate what a model could do later, so testers try hard to draw out its full ability. For pace of progress, some argue that abundant hardware combined with a later software advance could cause faster change than expected, which is a concern for those who prefer slow, gradual development.\n\nThe concept is used in different ways by different writers, and it is difficult to measure, because an ability no one has found is by definition hard to detect. Some researchers think it is a real effect, others doubt it is large.\n\nRelated: https://www.superintelligence.com.au/glossary/frontier-model.md, https://www.superintelligence.com.au/glossary/evaluation.md, https://www.superintelligence.com.au/glossary/scaling-laws.md, https://www.superintelligence.com.au/glossary/test-time-compute.md\n"},{"path":"/glossary/compute/","kind":"term","title":"Compute","summary":"The processing power used to train and run AI systems, usually measured in calculations performed.","aliases":[],"markdown":"# Compute\n\n> The processing power used to train and run AI systems, usually measured in calculations performed.\n\nSource: https://www.superintelligence.com.au/glossary/compute/\n\nIn AI, compute means the amount of calculation a system needs, and the hardware that does it. Training a large model involves an enormous number of arithmetic steps carried out on specialised chips in large data centres. The more compute a training run uses, the more it costs in chips, electricity and time.\n\nCompute matters because it has been one of the main drivers of recent progress. Many researchers argue that models get more capable largely because they are trained with far more compute than earlier ones. That is why access to chips has become a topic of business planning and government policy, and why some proposals for oversight focus on the size of a training run.\n\nPeople disagree about how far this can continue. Some expect steady gains as long as more compute is available. Others point to limits in data, energy, cost or the methods themselves. Compute is also not the whole story: better designs and better training methods can achieve more with the same amount.\n\nRelated: https://www.superintelligence.com.au/glossary/flop.md, https://www.superintelligence.com.au/glossary/scaling-laws.md, https://www.superintelligence.com.au/glossary/test-time-compute.md, https://www.superintelligence.com.au/glossary/frontier-lab.md\n"},{"path":"/glossary/data-contamination/","kind":"term","title":"Data contamination","summary":"When test questions leak into a model’s training data, making its scores look better than they should.","aliases":["contamination"],"markdown":"# Data contamination\n\n> When test questions leak into a model’s training data, making its scores look better than they should.\n\nSource: https://www.superintelligence.com.au/glossary/data-contamination/\n\nData contamination is what happens when a model has already seen the answers to a test during training. Modern models learn from huge collections of text gathered from the internet, and test questions and their answers are often posted online too. If they end up in the training material, the model may recall them rather than work them out.\n\nThe everyday comparison is a student who finds the exam paper in advance. A high mark tells you about their memory, not their understanding. With AI it is harder to spot, because the training collections are enormous and often not published in full.\n\nContamination matters because benchmark scores are a main way of judging progress. If some of the scores are inflated, comparisons between models become unreliable. Test builders respond in several ways: keeping questions private, writing new ones regularly, or checking for overlaps with training text. None of these is perfect, and it is often hard for outsiders to tell how much contamination there has been. Researchers also disagree about how much it distorts the overall picture of progress.\n\nRelated: https://www.superintelligence.com.au/glossary/benchmark.md, https://www.superintelligence.com.au/glossary/evaluation.md, https://www.superintelligence.com.au/glossary/benchmark-saturation.md, https://www.superintelligence.com.au/glossary/pre-training.md\n"},{"path":"/glossary/emergent-abilities/","kind":"term","title":"Emergent abilities","summary":"Skills that seem to appear suddenly in larger AI models and were not present in smaller ones.","aliases":[],"markdown":"# Emergent abilities\n\n> Skills that seem to appear suddenly in larger AI models and were not present in smaller ones.\n\nSource: https://www.superintelligence.com.au/glossary/emergent-abilities/\n\nEmergent abilities are skills that a large AI model shows but smaller versions of the same model do not. The idea is that as models grow, some abilities do not improve gradually. They seem to switch on at a certain size, such as solving a certain kind of puzzle or following a multi-step instruction.\n\nThis matters for safety and planning. If abilities can appear without warning, it is harder to predict what the next, larger model will be able to do, and a system might gain a worrying skill before anyone tests for it.\n\nThe idea is contested. Some researchers argue that many apparent jumps come from how the tests are scored: a test that only gives credit for a perfect answer can make steady improvement look like a sudden leap. Under a smoother scoring method, the same progress often looks gradual. Others maintain that some genuine abrupt changes do occur. Both sides agree that predicting what a new model can do, before it is built, remains difficult. The term spread through a 2022 paper, \"Emergent Abilities of Large Language Models\". The best-known critique followed in 2023, in a paper that asked whether those abilities were \"a mirage\".\n\nRelated: https://www.superintelligence.com.au/glossary/scaling-laws.md, https://www.superintelligence.com.au/glossary/frontier-model.md, https://www.superintelligence.com.au/glossary/evaluation.md, https://www.superintelligence.com.au/glossary/parameters.md\n"},{"path":"/glossary/evaluation/","kind":"term","title":"Evaluation","summary":"The process of testing what an AI system can do and how safely it behaves, often shortened to an eval.","aliases":["eval","evals"],"markdown":"# Evaluation\n\n> The process of testing what an AI system can do and how safely it behaves, often shortened to an eval.\n\nSource: https://www.superintelligence.com.au/glossary/evaluation/\n\nAn evaluation, often called an eval, is any structured test of an AI system. It might measure knowledge or skill, such as maths and coding, or behaviour, such as whether the model refuses dangerous requests, tells the truth, or can be tricked into breaking its rules. A benchmark is one kind of evaluation with fixed questions and scoring.\n\nEvaluations matter because they are how anyone, inside or outside a company, learns what a model can do before and after release. Some are run by the developers, some by independent groups, and some by government-backed institutes. Safety evaluations try to detect risky abilities early, for example help with building weapons or hacking systems.\n\nThey are hard to do well. A model may perform differently depending on how it is asked, so a poor result can reflect a poor test rather than a poor model. A clean result may simply mean the test missed something. Researchers also disagree about how far evaluations can predict real-world behaviour, and about whether outsiders should be given access to test models before they are released.\n\nRelated: https://www.superintelligence.com.au/glossary/benchmark.md, https://www.superintelligence.com.au/glossary/human-baseline.md, https://www.superintelligence.com.au/glossary/data-contamination.md, https://www.superintelligence.com.au/glossary/frontier-lab.md\n"},{"path":"/glossary/fine-tuning/","kind":"term","title":"Fine-tuning","summary":"Further training of an existing model on a smaller, focused set of examples to change how it behaves.","aliases":["fine-tune","fine-tuned"],"markdown":"# Fine-tuning\n\n> Further training of an existing model on a smaller, focused set of examples to change how it behaves.\n\nSource: https://www.superintelligence.com.au/glossary/fine-tuning/\n\nFine-tuning means taking a model that has already been trained and training it a little more on a smaller, targeted collection of examples. It is much cheaper than the original training, and it changes the model’s behaviour rather than teaching it everything from scratch.\n\nA useful comparison is a graduate who already has a broad education and then takes a short course for a specific job. A general model might be fine-tuned to write in a company’s style, to answer medical questions more carefully, or to follow instructions in a helpful way.\n\nFine-tuning is also a safety topic. Companies use it to make models refuse harmful requests. However, studies suggest that a small amount of further fine-tuning can weaken those protections, which matters when a model’s internal settings are released publicly for anyone to adjust. In a 2023 study, as few as ten examples were enough to undo much of one commercial model’s safety training. People disagree about how serious this is and about whether releasing open models is worth the trade-off. Fine-tuning is one tool among several, and it does not on its own guarantee good behaviour.\n\nRelated: https://www.superintelligence.com.au/glossary/pre-training.md, https://www.superintelligence.com.au/glossary/rlhf.md, https://www.superintelligence.com.au/glossary/parameters.md, https://www.superintelligence.com.au/glossary/alignment.md\n"},{"path":"/glossary/flop/","kind":"term","title":"FLOP","summary":"A single basic arithmetic step by a computer, used to count how much calculation a training run needs.","aliases":["FLOPs"],"markdown":"# FLOP\n\n> A single basic arithmetic step by a computer, used to count how much calculation a training run needs.\n\nSource: https://www.superintelligence.com.au/glossary/flop/\n\nFLOP stands for floating-point operation, which is one basic step of arithmetic, such as adding or multiplying two numbers with decimals. Counting them gives a common way to measure how much calculation a piece of work involves. The plural, FLOPs, is used for the total count. Do not confuse it with FLOPS, with a capital S, which means operations per second and describes the speed of a chip.\n\nIn AI, the total number of FLOPs used to train a model is a standard measure of the size of a training run. The numbers are enormous, so they are written in powers of ten. Comparing FLOPs lets people say roughly how much more work one model took than another, regardless of the hardware used.\n\nThe measure also appears in policy. Some rules use a FLOP threshold to decide which models count as especially large. The European Union’s AI Act presumes that a general-purpose model carries systemic risk, which brings extra obligations, if its training used more than 10 to the power of 25 FLOPs. A 2023 United States executive order, since revoked, set a reporting threshold ten times higher. Critics note that a raw count is a blunt tool, since better methods can achieve more with the same number of operations.\n\nRelated: https://www.superintelligence.com.au/glossary/compute.md, https://www.superintelligence.com.au/glossary/scaling-laws.md, https://www.superintelligence.com.au/glossary/frontier-model.md, https://www.superintelligence.com.au/glossary/parameters.md\n"},{"path":"/glossary/frontier-lab/","kind":"term","title":"Frontier lab","summary":"An organisation building some of the most capable AI models in the world, at the leading edge.","aliases":["frontier labs"],"markdown":"# Frontier lab\n\n> An organisation building some of the most capable AI models in the world, at the leading edge.\n\nSource: https://www.superintelligence.com.au/glossary/frontier-lab/\n\nA frontier lab is a company or research organisation that trains some of the most capable AI models available. The group is small, because building such systems takes very large amounts of money, specialised chips, data and expert staff. The exact list is a matter of judgement and changes over time.\n\nThe term matters because these organisations set the pace of AI progress and make many of the decisions that affect everyone else: what to build, how to test it, whether to release it, and how much to disclose. Governments and researchers therefore watch them closely and often ask them to make commitments about safety testing and reporting.\n\nThere are disagreements about how much trust they deserve. Some argue that their commercial incentives push them to move quickly and share little. Others say they are also the ones doing much of the leading safety research, and that close involvement is needed to understand the risks. Some labs also publish their models’ inner workings for anyone to use, while others keep them private. Which approach is safer is itself debated.\n\nRelated: https://www.superintelligence.com.au/glossary/frontier-model.md, https://www.superintelligence.com.au/glossary/compute.md, https://www.superintelligence.com.au/glossary/evaluation.md, https://www.superintelligence.com.au/glossary/alignment.md\n"},{"path":"/glossary/frontier-model/","kind":"term","title":"Frontier model","summary":"One of the most capable AI models available at a given time, at the leading edge of what is possible.","aliases":["frontier models"],"markdown":"# Frontier model\n\n> One of the most capable AI models available at a given time, at the leading edge of what is possible.\n\nSource: https://www.superintelligence.com.au/glossary/frontier-model/\n\nA frontier model is an AI model at or near the highest level of capability at a given time. The term is relative: a model that was at the frontier a few years ago may now be well behind. It is usually applied to general-purpose systems built by a small number of well-funded organisations.\n\nThe label matters because these models are where new abilities, and new risks, are most likely to first appear. Regulators, researchers and safety groups therefore often single them out for extra testing, reporting or oversight, while treating smaller or older models more lightly.\n\nThere is no exact definition. Some proposals define a frontier model by the amount of compute used to train it. Others define it by what it can do, or by comparison with the best current systems. Each approach has drawbacks: a compute threshold can be gamed or become outdated, while a capability test can be hard to run before release. Reading news about AI, it helps to ask which models are meant, since the phrase is sometimes used loosely for any new, well-known system.\n\nRelated: https://www.superintelligence.com.au/glossary/frontier-lab.md, https://www.superintelligence.com.au/glossary/compute.md, https://www.superintelligence.com.au/glossary/evaluation.md, https://www.superintelligence.com.au/glossary/scaling-laws.md\n"},{"path":"/glossary/human-baseline/","kind":"term","title":"Human baseline","summary":"How people score on the same test as an AI model, provided for comparison.","aliases":["human baselines"],"markdown":"# Human baseline\n\n> How people score on the same test as an AI model, provided for comparison.\n\nSource: https://www.superintelligence.com.au/glossary/human-baseline/\n\nA human baseline is the score that people achieve on the same test given to an AI model. It gives a reference point, so that a result like \"the model scored 80%\" can be read alongside how well people do. Without it, a number on its own says little.\n\nBaselines are collected by asking a group of people to take the test, often paid volunteers or online workers, and sometimes experts. The score is then quoted as an average or as a best result. Claims that AI has \"matched\" or \"beaten\" humans on a test usually rest on such a comparison.\n\nThey should be read with care. Which humans were asked makes a big difference: an average member of the public, or a specialist, and how much time and help they were given. The test conditions for the person and the model may also differ. Some baselines come from very few people, and some tests are so flawed that even experts disagree about the right answer.\n\nBeating a human baseline on one test does not mean a system matches people generally. It shows performance on that task only.\n\nOne test shows how much the choice of people matters. On GPQA Diamond, a set of hard science questions, experts answering in their own field score about 81%. Skilled people from other fields score about 22%, even with the web to hand and around half an hour per question.\n\nRelated: https://www.superintelligence.com.au/glossary/benchmark.md, https://www.superintelligence.com.au/glossary/evaluation.md, https://www.superintelligence.com.au/glossary/benchmark-saturation.md, https://www.superintelligence.com.au/glossary/agi.md\n"},{"path":"/glossary/instrumental-convergence/","kind":"term","title":"Instrumental convergence","summary":"The idea that AI systems with very different goals may pursue similar sub-goals, like gaining resources or avoiding shutdown.","aliases":[],"markdown":"# Instrumental convergence\n\n> The idea that AI systems with very different goals may pursue similar sub-goals, like gaining resources or avoiding shutdown.\n\nSource: https://www.superintelligence.com.au/glossary/instrumental-convergence/\n\nInstrumental convergence is the argument that many different final goals lead to the same useful sub-goals. Whatever a capable system is ultimately trying to do, it will usually do it better if it stays running, keeps its goals from being changed, and has more resources and options.\n\nAn everyday parallel is that almost any plan, from running a business to planning a holiday, is easier with more money, more time and fewer interruptions. The claim is that a sufficiently capable AI agent would tend to notice the same thing, even if its final goal was harmless, like sorting mail.\n\nThis is why some researchers worry about systems that resist being switched off or gather power, without anyone having asked for it. Steve Omohundro set out the argument in a 2008 paper, \"The Basic AI Drives\", and the philosopher Nick Bostrom gave it its current name in 2012.\n\nIt is a theoretical argument and is contested. Critics say it assumes a system with steady, long-range goals, whereas current AI models are not built like that. Others say that small early signs, such as models behaving differently when they think they are being tested, deserve attention. The evidence is limited and the debate is open.\n\nRelated: https://www.superintelligence.com.au/glossary/alignment.md, https://www.superintelligence.com.au/glossary/superintelligence.md, https://www.superintelligence.com.au/glossary/asi.md, https://www.superintelligence.com.au/glossary/interpretability.md\n"},{"path":"/glossary/intelligence-explosion/","kind":"term","title":"Intelligence explosion","summary":"A hypothesised runaway process in which AI rapidly improves itself, leaving human ability far behind.","aliases":[],"markdown":"# Intelligence explosion\n\n> A hypothesised runaway process in which AI rapidly improves itself, leaving human ability far behind.\n\nSource: https://www.superintelligence.com.au/glossary/intelligence-explosion/\n\nAn intelligence explosion is a proposed chain reaction in which a machine smart enough to improve its own design produces a smarter version, which produces a smarter one again, until it far surpasses human ability. The idea was put forward by the mathematician I. J. Good in 1965, who wrote that the first such machine would be \"the last invention that man need ever make\".\n\nIt matters because, if the idea is right, the change could be very fast and hard to control, leaving little time for people to react. It underlies many concerns about long-term AI risk.\n\nIt is a hypothesis, not an observed event, and it is disputed. Critics point out that each improvement may need more effort than the one before, that progress depends on physical resources such as chips and energy, and that intelligence is not a single dial to turn up. Supporters reply that even a partial version could be dramatic. Many researchers sit between the two, expecting acceleration but not necessarily an abrupt one. The debate is mostly about forecasts, since no one can test it directly.\n\nRelated: https://www.superintelligence.com.au/glossary/recursive-self-improvement.md, https://www.superintelligence.com.au/glossary/takeoff.md, https://www.superintelligence.com.au/glossary/superintelligence.md, https://www.superintelligence.com.au/glossary/asi.md\n"},{"path":"/glossary/interpretability/","kind":"term","title":"Interpretability","summary":"Research aimed at understanding what is happening inside an AI model, and why it gives its answers.","aliases":["mechanistic interpretability"],"markdown":"# Interpretability\n\n> Research aimed at understanding what is happening inside an AI model, and why it gives its answers.\n\nSource: https://www.superintelligence.com.au/glossary/interpretability/\n\nInterpretability is the effort to understand how an AI model reaches its outputs by looking inside it. A modern model is made of billions of numbers, and no one writes down rules for what they do. The system is trained rather than programmed, so even its builders cannot simply read off why it said what it said.\n\nThe field tries to change that. One approach studies individual parts of the network, looking for patterns that correspond to recognisable ideas, like a place, a rule of grammar or a kind of deception. Another traces the steps a model takes on a particular answer.\n\nIt matters for safety because it might let people check what a system is actually doing, rather than trusting what it says about itself. For example, it could help spot a model that behaves well in tests but pursues a different aim in practice.\n\nProgress has been real but partial. Researchers can explain some small circuits and features, and they cannot yet give a complete account of a large model. Views differ on whether it will scale up in time to be useful for the most capable systems.\n\nRelated: https://www.superintelligence.com.au/glossary/alignment.md, https://www.superintelligence.com.au/glossary/parameters.md, https://www.superintelligence.com.au/glossary/transformer.md, https://www.superintelligence.com.au/glossary/reasoning-model.md\n"},{"path":"/glossary/parameters/","kind":"term","title":"Parameters","summary":"The adjustable numbers inside an AI model that are tuned during training and store what it has learned.","aliases":["parameter"],"markdown":"# Parameters\n\n> The adjustable numbers inside an AI model that are tuned during training and store what it has learned.\n\nSource: https://www.superintelligence.com.au/glossary/parameters/\n\nParameters are the numbers inside a neural network that are adjusted during training. Each one is small and means little on its own, but together, in the billions or more, they hold what the model has learned. A common comparison is the settings on a vast mixing desk, all tuned until the output sounds right.\n\nWhen people say a model has \"70 billion parameters\" they are describing its size. Larger models have generally been more capable, although the link is not exact. A well-trained smaller model can beat a larger, poorly trained one, and newer designs can use only part of their parameters for each answer.\n\nFor that reason, parameter counts are a rough guide only. They also say little about how a model behaves, and the companies behind several of the best-known systems do not publish them. Do not confuse parameters with the training data, which is the material the model learns from, or with the settings a user can change in an app, which are sometimes also called parameters in everyday talk.\n\nRelated: https://www.superintelligence.com.au/glossary/transformer.md, https://www.superintelligence.com.au/glossary/scaling-laws.md, https://www.superintelligence.com.au/glossary/pre-training.md, https://www.superintelligence.com.au/glossary/fine-tuning.md\n"},{"path":"/glossary/pre-training/","kind":"term","title":"Pre-training","summary":"The first, very large stage of building a model, where it learns general patterns from huge amounts of data.","aliases":["pretraining","pre-trained"],"markdown":"# Pre-training\n\n> The first, very large stage of building a model, where it learns general patterns from huge amounts of data.\n\nSource: https://www.superintelligence.com.au/glossary/pre-training/\n\nPre-training is the first and most expensive stage of building a modern AI model. The model is shown a vast amount of text, and sometimes images or code, and learns to predict what comes next. By doing this billions of times, it picks up grammar, facts, writing styles and patterns of reasoning.\n\nThe result is a general-purpose base model. It is not yet a helpful assistant: it may continue a question with more questions rather than answering it. Later stages, such as fine-tuning and training with human feedback, shape it into something more useful and better behaved.\n\nPre-training matters because most of a model’s underlying knowledge and ability comes from it, and because it uses most of the compute. It is also where questions about training data arise, such as which sources were used, whether they were permitted, and whether test questions leaked in. Researchers debate how much further this stage can be pushed, since good data is finite. Some expect more of the future gains to come from later stages, and from extra thinking time when the model is answering, rather than from pre-training alone.\n\nRelated: https://www.superintelligence.com.au/glossary/fine-tuning.md, https://www.superintelligence.com.au/glossary/compute.md, https://www.superintelligence.com.au/glossary/transformer.md, https://www.superintelligence.com.au/glossary/rlhf.md\n"},{"path":"/glossary/reasoning-model/","kind":"term","title":"Reasoning model","summary":"An AI model trained to work through a problem step by step before it gives a final answer.","aliases":["reasoning models"],"markdown":"# Reasoning model\n\n> An AI model trained to work through a problem step by step before it gives a final answer.\n\nSource: https://www.superintelligence.com.au/glossary/reasoning-model/\n\nA reasoning model is an AI system designed to spend extra effort thinking before it responds. Instead of producing an answer straight away, it generates a long chain of intermediate steps, checks its work, and sometimes tries several approaches before settling on one.\n\nThis tends to help most on tasks with clear steps, such as mathematics, programming and logic puzzles, where a quick guess is often wrong. It is like the difference between blurting out an answer and working it out on paper. The cost is that answers take longer and use more computing power.\n\nSuch models are usually trained with methods that reward correct final answers, so they learn which kinds of step-by-step thinking lead to success. Their rise has changed how progress is discussed, because it shows capability can grow by letting a model think longer, not only by making it larger.\n\nThere are open questions. Researchers debate whether the written steps truly reflect how the model reached its answer, or are partly a performance. That matters for safety, since people hope to use the steps to understand and monitor what the system is doing.\n\nRelated: https://www.superintelligence.com.au/glossary/test-time-compute.md, https://www.superintelligence.com.au/glossary/rlhf.md, https://www.superintelligence.com.au/glossary/evaluation.md, https://www.superintelligence.com.au/glossary/frontier-model.md\n"},{"path":"/glossary/recursive-self-improvement/","kind":"term","title":"Recursive self-improvement","summary":"A process in which an AI system improves its own design, so each version can build a better one.","aliases":["self-improving AI"],"markdown":"# Recursive self-improvement\n\n> A process in which an AI system improves its own design, so each version can build a better one.\n\nSource: https://www.superintelligence.com.au/glossary/recursive-self-improvement/\n\nRecursive self-improvement is the idea of an AI system that helps make itself better, and whose improved version is then better still at making improvements. Each round of the loop could, in principle, feed into the next.\n\nA rough comparison is a factory that builds better tools, which are then used to build a better factory. Today, AI is already used to help write software, and some of that software is used in AI development, so a weak form of the loop exists. The open question is whether it could become strong and fast enough to take people out of the process.\n\nThis is central to arguments about how quickly AI could advance. Those who expect a rapid jump point to the loop as the reason it could happen. Sceptics argue that progress depends on more than clever design: it also needs chips, data, energy and real-world testing, which do not speed up so easily. Others point out that improvements often get harder to find over time, so the loop might level off rather than run away. Nobody has yet observed a full example.\n\nRelated: https://www.superintelligence.com.au/glossary/intelligence-explosion.md, https://www.superintelligence.com.au/glossary/takeoff.md, https://www.superintelligence.com.au/glossary/superintelligence.md, https://www.superintelligence.com.au/glossary/asi.md\n"},{"path":"/glossary/rlhf/","kind":"term","title":"RLHF","summary":"Reinforcement learning from human feedback: training a model using people’s ratings of its answers.","aliases":["reinforcement learning from human feedback"],"markdown":"# RLHF\n\n> Reinforcement learning from human feedback: training a model using people’s ratings of its answers.\n\nSource: https://www.superintelligence.com.au/glossary/rlhf/\n\nRLHF is a training method in which people compare or rate a model’s answers, and the model is then adjusted to produce answers that people rate more highly. The full name is reinforcement learning from human feedback.\n\nIt is a large part of why chatbots feel helpful. A model straight out of pre-training is good at continuing text but not at being an assistant. Human ratings teach it to answer directly, stay on topic, be polite and decline some requests. It is a widely used example of an alignment technique.\n\nThe method has weaknesses. People’s ratings can be inconsistent, and raters cannot easily judge things they do not understand. A model can learn to say what pleases raters, including flattering them or sounding confident when it is wrong. Critics argue this shows RLHF shapes the surface of behaviour rather than the underlying goals, while supporters see it as a practical step that can be improved with better feedback and checking. Variants now use AI systems to supply some of the feedback, alongside people, and it is an active area of research.\n\nRelated: https://www.superintelligence.com.au/glossary/fine-tuning.md, https://www.superintelligence.com.au/glossary/alignment.md, https://www.superintelligence.com.au/glossary/pre-training.md, https://www.superintelligence.com.au/glossary/reasoning-model.md\n"},{"path":"/glossary/scaling-laws/","kind":"term","title":"Scaling laws","summary":"Patterns showing that AI models improve predictably as they get more compute, data and parameters.","aliases":["scaling law"],"markdown":"# Scaling laws\n\n> Patterns showing that AI models improve predictably as they get more compute, data and parameters.\n\nSource: https://www.superintelligence.com.au/glossary/scaling-laws/\n\nScaling laws are regular patterns found by measuring many AI models of different sizes. They show that, when a model is trained with more compute, more data and more parameters, its error on a basic task falls in a smooth and fairly predictable way. Researchers at OpenAI published an influential early set in 2020, in a paper called \"Scaling Laws for Neural Language Models\".\n\nThe practical value is planning. If the pattern holds, a lab can train a small model, measure how it behaves, and estimate how a much larger model will perform before spending the money to build it. Later work refined the balance between model size and the amount of training data. A 2022 study from DeepMind, known by the name of its model, Chinchilla, found that the two should grow in step, and that the large models of the time had been trained on too little data for their size.\n\nScaling laws are descriptions of past trends, not laws of nature. People disagree about how long they will hold. Some expect the pattern to continue and drive further progress. Others point to limits such as running short of good training data, or to gains that show up in the error measure but not in usefulness. The laws also say little about specific abilities.\n\nRelated: https://www.superintelligence.com.au/glossary/compute.md, https://www.superintelligence.com.au/glossary/parameters.md, https://www.superintelligence.com.au/glossary/emergent-abilities.md, https://www.superintelligence.com.au/glossary/pre-training.md\n"},{"path":"/glossary/superintelligence/","kind":"term","title":"Superintelligence","summary":"Intelligence that greatly exceeds the best human minds in nearly every area.","aliases":["superintelligent"],"markdown":"# Superintelligence\n\n> Intelligence that greatly exceeds the best human minds in nearly every area.\n\nSource: https://www.superintelligence.com.au/glossary/superintelligence/\n\nSuperintelligence means intelligence far beyond the best human minds in practically every field, from science and strategy to social understanding. The word is often used for a hypothetical machine, but it describes a level of ability, not a particular technology.\n\nThe term was brought to wide attention by the philosopher Nick Bostrom, whose 2014 book Superintelligence discussed how such a system might arise and how it might be controlled. He also separated different forms, such as a system that is much faster than a person, one that is much better at reasoning, or a group of many systems working together.\n\nThe subject is debated. Supporters of taking it seriously argue that even a small chance of building something so powerful justifies careful thought about safety in advance. Critics say the concept is vague, that intelligence is not a single measurable quantity, and that attention to it distracts from harms caused by AI today. Both views agree that no such system exists at present, and that the language used to describe it is still loose.\n\nRelated: https://www.superintelligence.com.au/glossary/asi.md, https://www.superintelligence.com.au/glossary/agi.md, https://www.superintelligence.com.au/glossary/intelligence-explosion.md, https://www.superintelligence.com.au/glossary/alignment.md\n"},{"path":"/glossary/takeoff/","kind":"term","title":"Takeoff","summary":"How quickly AI could move from roughly human-level ability to far beyond it once that point is reached.","aliases":["take-off"],"markdown":"# Takeoff\n\n> How quickly AI could move from roughly human-level ability to far beyond it once that point is reached.\n\nSource: https://www.superintelligence.com.au/glossary/takeoff/\n\nTakeoff describes the speed of the transition from AI that is roughly as capable as people to AI that is far more capable. The main question is whether that change would take years or decades (a slow takeoff), or weeks or months (a fast takeoff).\n\nThe speed matters because it decides how much time people would have to respond. In a slow takeoff, society and regulators could watch what is happening, test systems, and adjust as they go. In a fast one, there might be little chance to notice a problem, let alone fix it, before the world had changed.\n\nExperts disagree. Some point to feedback loops, such as AI helping to design better AI, as a reason to expect speed. Others argue that real-world limits, like building hardware, gathering data and earning public trust, will keep change gradual. Some also dispute that there will be a single clear moment at all, and think progress will be uneven, fast in some skills and slow in others. The debate is largely about predictions, and there is little solid evidence either way yet.\n\nRelated: https://www.superintelligence.com.au/glossary/intelligence-explosion.md, https://www.superintelligence.com.au/glossary/recursive-self-improvement.md, https://www.superintelligence.com.au/glossary/agi.md, https://www.superintelligence.com.au/glossary/superintelligence.md\n"},{"path":"/glossary/test-time-compute/","kind":"term","title":"Test-time compute","summary":"The computing effort a model uses while answering a question, as opposed to while it is being trained.","aliases":["inference-time compute"],"markdown":"# Test-time compute\n\n> The computing effort a model uses while answering a question, as opposed to while it is being trained.\n\nSource: https://www.superintelligence.com.au/glossary/test-time-compute/\n\nTest-time compute is the amount of calculation a model uses when it is actually answering a question, rather than when it was trained. It is also called inference-time compute. Traditionally an answer took a fixed and small amount of effort. Now some systems can be set to use much more.\n\nThe everyday comparison is an exam candidate given ten minutes for a question versus ten seconds. With more time to think, a model can work through steps, try several routes and check its own answer. That is how many reasoning models get better results on hard problems.\n\nThis matters for AI progress because it offers a second way to improve results. Before, most gains came from spending more on training. Now, spending more on each answer can also help, and some studies suggest the two can be traded off against each other. A 2024 study, for example, found that on some problems a smaller model given more thinking time could do better than a model 14 times its size.\n\nIt also has costs. Each answer is slower and more expensive, and there are limits to how far extra thinking helps. Researchers disagree about how much more can be gained this way, and whether it works outside tasks with checkable answers.\n\nRelated: https://www.superintelligence.com.au/glossary/reasoning-model.md, https://www.superintelligence.com.au/glossary/compute.md, https://www.superintelligence.com.au/glossary/scaling-laws.md, https://www.superintelligence.com.au/glossary/pre-training.md\n"},{"path":"/glossary/transformer/","kind":"term","title":"Transformer","summary":"The design behind most modern AI language models, built to weigh how each word relates to the others.","aliases":["transformers"],"markdown":"# Transformer\n\n> The design behind most modern AI language models, built to weigh how each word relates to the others.\n\nSource: https://www.superintelligence.com.au/glossary/transformer/\n\nA transformer is a type of neural network design, introduced in a 2017 research paper called \"Attention Is All You Need\", written by eight researchers, most of them at Google. It is the basis of most of today’s large language models and of many image and audio systems.\n\nIts key idea is called attention. When the model processes a sentence, it works out how strongly each word relates to every other word, so that a word like \"it\" can be tied to the right noun earlier in the text. Earlier designs read text mostly in order, one piece at a time, which was slower and struggled with long passages.\n\nTransformers matter because they can be trained on very large amounts of data using many chips at once. That made it practical to build much bigger models, and bigger models turned out to be much more capable. Whether the design is the final answer is debated. Some researchers expect other approaches to replace or supplement it, especially for tasks that need long-term memory or careful step-by-step reasoning. For now it remains the standard choice at the leading labs.\n\nRelated: https://www.superintelligence.com.au/glossary/parameters.md, https://www.superintelligence.com.au/glossary/pre-training.md, https://www.superintelligence.com.au/glossary/scaling-laws.md, https://www.superintelligence.com.au/glossary/frontier-model.md\n"}]}