Transformer
The design behind most modern AI language models, built to weigh how each word relates to the others.
A transformer is a type of neural network design, introduced in a 2017 research paper called “Attention Is All You Need”, written by eight researchers, most of them at Google. It is the basis of most of today’s large language models and of many image and audio systems.
Its key idea is called attention. When the model processes a sentence, it works out how strongly each word relates to every other word, so that a word like “it” can be tied to the right noun earlier in the text. Earlier designs read text mostly in order, one piece at a time, which was slower and struggled with long passages.
Transformers matter because they can be trained on very large amounts of data using many chips at once. That made it practical to build much bigger models, and bigger models turned out to be much more capable. Whether the design is the final answer is debated. Some researchers expect other approaches to replace or supplement it, especially for tasks that need long-term memory or careful step-by-step reasoning. For now it remains the standard choice at the leading labs.