Glossary

28 terms, each in a sentence and then a little more.

A

AGI: Artificial general intelligence: AI that can handle a wide range of tasks at roughly human level.
Alignment: The work of making AI systems pursue the goals their designers and users actually intend.
ASI: Artificial superintelligence: AI that would greatly outperform the best humans at nearly all intellectual tasks.

Benchmark: A standard set of test questions used to compare how well different AI models perform.
Benchmark saturation: When top models score close to the maximum on a test, so it can no longer tell them apart.

Capability overhang: When an AI system can do more than people have yet realised, or been able to draw out of it.
Compute: The processing power used to train and run AI systems, usually measured in calculations performed.

Data contamination: When test questions leak into a model’s training data, making its scores look better than they should.

Emergent abilities: Skills that seem to appear suddenly in larger AI models and were not present in smaller ones.
Evaluation: The process of testing what an AI system can do and how safely it behaves, often shortened to an eval.

Fine-tuning: Further training of an existing model on a smaller, focused set of examples to change how it behaves.
FLOP: A single basic arithmetic step by a computer, used to count how much calculation a training run needs.
Frontier lab: An organisation building some of the most capable AI models in the world, at the leading edge.
Frontier model: One of the most capable AI models available at a given time, at the leading edge of what is possible.

Human baseline: How people score on the same test as an AI model, provided for comparison.

Instrumental convergence: The idea that AI systems with very different goals may pursue similar sub-goals, like gaining resources or avoiding shutdown.
Intelligence explosion: A hypothesised runaway process in which AI rapidly improves itself, leaving human ability far behind.
Interpretability: Research aimed at understanding what is happening inside an AI model, and why it gives its answers.

Parameters: The adjustable numbers inside an AI model that are tuned during training and store what it has learned.
Pre-training: The first, very large stage of building a model, where it learns general patterns from huge amounts of data.

Reasoning model: An AI model trained to work through a problem step by step before it gives a final answer.
Recursive self-improvement: A process in which an AI system improves its own design, so each version can build a better one.
RLHF: Reinforcement learning from human feedback: training a model using people’s ratings of its answers.

Scaling laws: Patterns showing that AI models improve predictably as they get more compute, data and parameters.
Superintelligence: Intelligence that greatly exceeds the best human minds in nearly every area.

Takeoff: How quickly AI could move from roughly human-level ability to far beyond it once that point is reached.
Test-time compute: The computing effort a model uses while answering a question, as opposed to while it is being trained.
Transformer: The design behind most modern AI language models, built to weigh how each word relates to the others.