Skip to content
AI Universe
Français

Step 5 of 10

Meaning

How can a machine tell that "cat" is close to "dog"?

Make a guess

On the map of meaning this site shows, which of these two words is closer to "cat"?

  • "dog", another pet.
  • "chat", the same animal in French.

Show the answer

The answer: "chat", the same animal in French.

The map sorts words by what they mean, not by their language. "cat" and "chat" name the same animal, so they land almost on the same spot. "dog" is in the same corner, with the animals, but farther away.

What happens

  1. A token's number says nothing about what it means. So the model swaps each number for a long list of numbers: a position in a huge space, a bit like an address.

  2. Those positions are learned. The more the model reads, the closer it moves words that turn up in the same kinds of sentences.

  3. That space has hundreds of dimensions, and nobody can draw it. The map in this step comes from a smaller model than Qwen3-4B, one built to sort texts by meaning, flattened down to three dimensions.

  4. On this map, "Why is the sky blue?" lands among these words: Computed for this site by the embedding model multilingual-e5-small, on November 7, 2026.Question asked: "Why is the sky blue?"The six words on the map closest to this question, in the model's original space, before it is flattened to three dimensions.Model card.

  5. "bleu" is French, yet no word on the map sits closer to "blue": words are placed by meaning.

Your turn

Pick one of the site's questions and see where it lands on the map, and which words surround it. Then zoom out to the whole map: words on the same topic form neighborhoods, in English and in French alike.

Why is the sky blue?

The map, flattened

the questionbluebleusunespacestarrocketsnowvolcanoétérainmountaincoldatmospherehotplage

The six words closest to the question

  • translation: ciel / sky
  • bleu / blue
  • translation: nuage / cloud
  • translation: soleil / sun
  • translation: espace / space

Measured in the model's original space, with 384 dimensions. The neighbors are in bold, next to their translation.

The map loses neighbors. In the original space, a word's translation is among its three closest neighbors 78% of the time; on the 3D map, 51%; on this flat view, 35%.

Embeddings computed by the Xenova/multilingual-e5-small model, revision 761b726, and laid out with UMAP.

Takeaway

Each token becomes a long list of numbers, a position in a space with hundreds of dimensions. Words with similar meanings end up close together, whatever their language.

Go further

A map that distorts

Squeezing hundreds of dimensions into three loses some neighbors: on the map, a word's translation is less often among its closest neighbors than in the original space. The model itself sometimes pulls together words that are spelled almost the same. The question about Mars lands near Computed for this site by the embedding model multilingual-e5-small, on November 7, 2026.Question asked: "How long does it take to get to Mars?"The six words on the map closest to this question, in the model's original space, before it is flattened to three dimensions.Model card. "mer" (sea) and "marché" (market) are French words that are probably there for their letters more than their meaning.

Prejudice, stored with the rest

The positions come from texts, prejudice included. In one study, models of the same kind as the one behind this map ranked identical resumes where only the first name changed. In Tests where, with identical resumes, a White-associated name is ranked ahead of a Black-associated nameThree embedding models, nine occupations, over 500 resumes and 500 job descriptions, over three million comparisons; the advantage goes to Black-associated names in 8.6% of testsSource: Wilson and Caliskan, Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval (arXiv 2407.20371) (July 29, 2024)Read at the source, from the organization that produced it., a name associated with White people came out ahead of a name associated with Black people.

Inside Qwen3-4B

The model from the other steps does the same thing on a bigger scale: each of its tokens becomes a list of thousands of numbers. Attention then reworks those starting positions, in the next step ("Attention").

Sources