Insight · September 22, 2026
What are
embeddings.
Every AI system that seems to understand meaning starts here. An embedding turns a word into a list of numbers and puts similar meanings in the same place.
01 · The idea
Meaning becomes a location.
A computer cannot read. It matches characters. The word refund and the phrase get my money back share almost no letters, yet they mean the same thing, and exact matching will never connect them.
An embedding closes that gap. It takes a piece of text and returns a list of numbers. That list is a set of coordinates. It places the meaning of the text at a single point in a large space, and it places text that means something similar nearby. Refund and get my money back land close together, even though they look nothing alike. That is the entire trick, and almost every AI feature that feels like it understands you rests on it.
02 · How it works
How a list of numbers can hold meaning.
Picture a map. Two numbers, a latitude and a longitude, are enough to place any city on Earth. Cities that sit near each other tend to share weather, food, and accent. Position carries information.
An embedding does the same thing with meaning instead of geography, and it uses far more than two numbers. The text-embedding-3-small model from OpenAI returns 1,536 numbers for every input. Its larger model returns 3,072. Each number is one axis in a space no person can picture, and the model learned during training where to put things so that related meanings sit close.
You never set the numbers by hand. The model reads enormous amounts of text and works out the coordinates on its own. That is what makes an embedding different from a label someone typed. Nobody decided that refund and get my money back belong together. The model placed them there because that is how they are used.
The one line to keep
“An embedding turns meaning into a place. Similar things sit close, and closeness is something a computer can measure.”
03 · Where it came from
The idea is older than the chatbot.
In 2013 a group of Google researchers led by Tomas Mikolov published a method called word2vec, in a paper titled Efficient Estimation of Word Representations in Vector Space. It learned a vector for each word by predicting the words around it. It came in two forms. One predicted a word from its neighbors, the other predicted the neighbors from a word, and both landed in the same kind of space.
A year later, in 2014, Jeffrey Pennington, Richard Socher, and Christopher Manning at Stanford released GloVe, which reached the same kind of result a different way. Rather than predicting, it counted how often words appear together across a body of text and factored those counts into vectors. Their public vectors were trained on six billion words and covered a vocabulary of 400,000 words, offered in sizes from 50 to 300 numbers each. Predicting or counting, the destination was the same.
Both proved the same surprising fact. Feed a model nothing but raw text and it will arrange words into a space where distance means similarity, with no one labeling anything. Today's models are larger and read sentences in context rather than one word at a time, but the shape of the idea has not changed.
04 · The famous result
King minus man plus woman.
The result that made embeddings famous is a piece of arithmetic. Take the vector for king, subtract the vector for man, add the vector for woman, and the closest point in the space is queen. The direction that separates man from woman is roughly the same direction that separates king from queen, so the space encodes not just similarity but relationships.
It is worth being precise, because the demo is often oversold. Queen is the nearest word only after the three input words are set aside. Leave them in and the nearest point is king itself. The relationship is real. It is just quieter than the headline suggests, and that honesty matters when you are deciding what to trust this layer to do.
05 · What it powers
Closeness is a number you can compute.
Once every piece of text is a point, comparing two of them is arithmetic. The common measure is cosine similarity, the angle between two vectors. A small angle means the meanings point the same way. Other measures exist, like straight line distance and the dot product, but they answer the same question: how close are these two meanings. Search stops being about matching words and becomes about matching direction. That one move is what sits under the features you already use.
01
Semantic search
The query becomes a point, the documents are already points, and the closest ones come back. It finds the right passage even when it shares no words with what you typed.
02
Retrieval augmented generation
Before the model answers, the system embeds the question, pulls the nearest passages from your own documents, and hands them over as grounding.
03
Recommendations
Items whose points sit near each other are the ones people who liked one tend to like the other. No manual tagging required.
04
Clustering and deduplication
Points that bunch together are near duplicates or a shared theme, surfaced without a single hand written rule.
06 · A common mix up
An embedding is not a token.
These two words get swapped, and they are not the same step. A token is a unit of text. Before a model reads anything, it splits the input into tokens, whole words or fragments of words, and each token is an entry in a fixed vocabulary with a number for an index. That index is just a name. It carries no meaning.
The embedding is the next step. It takes each token and computes its vector, the list of numbers that actually holds the meaning. Tokenizing tells you which pieces you have. Embedding tells you what they mean. One comes before the other, and only the second lets a system reason about similarity.
07 · Why it matters
This layer sits under everything.
For a business, the embedding is the quiet layer that decides whether an AI feature feels sharp or dumb. If your search cannot connect a refund to a customer asking for their money back, the model on top never sees the right document, and no amount of prompting saves the answer.
Choosing an embedding model is a real decision. Larger vectors capture more but cost more to store and compare. Modern models let you trade the two on purpose. OpenAI's larger model can be shortened from 3,072 numbers down to 256 and still outperform the previous generation at its full size, which means you can cut storage without starting over.
The point is not the specific numbers. It is that this layer is a choice, not a default, and the quality of everything above it depends on getting it right.
Closing
Meaning becomes a place.
Pick one search box in your product that still matches words. Embed the queries and the content, rank by distance instead of overlap, and watch it start finding what people meant rather than what they typed. That is the whole shift, and it is one layer down from everything else you are building.
Written by · September 22, 2026
Donny Smith · ECD, Founder, Bttr.
Over the past 15 years, he has led creative teams and contributed to products used by millions of people worldwide, working with companies including Apple, Opendoor, JP Morgan, GE Aerospace, Pepsi, and Alterra Mountain Company.
LinkedInword2vec · Mikolov et al., Efficient Estimation of Word Representations in Vector Space, Google, 2013 · GloVe · Pennington, Socher, Manning, Stanford, 2014 · dimensions and shortening behavior from OpenAI's text embedding 3 model notes · analogy behavior per the original word2vec evaluation
Share this perspective
More insights
Adjacent perspectives.
Bttr. Field Brief
The brief Bttr. writes for senior buyers.
Monthly. One signal worth your time on Brand Operating Systems, AI search visibility, and the infrastructure buildout. No filler.