Skip to content
AI Universe
Français

Step 8 of 10

Where its knowledge comes from

Who taught the model to talk like this?

Make a guess

The model answers your questions politely, like an assistant. Who taught it to do that?

  • Nobody: it picked that up on its own from the texts it read.
  • People, who wrote examples and rated its answers.
  • A programmer, who wrote an answer in advance for every question.

Show the answer

The answer: People, who wrote examples and rated its answers.

Reading texts teaches the model to continue them, not to answer like an assistant. For that, people wrote examples of good answers and compared the model's answers, a huge number of times.

What happens

  1. Before it answered anyone, the model read. A lot. During its pre-training, Qwen3-4B went through Tokens of text read by the Qwen3 models, Qwen3-4B included, during pre-trainingText in 119 languages and dialects; the report gives the kinds of text, not where each one comes fromSource: Qwen Team, Qwen3 Technical Report (arXiv 2505.09388) (May 14, 2025)Read at the source, from the organization that produced it..

  2. Its only exercise: guess the next token, check against the real text, nudge its billions of internal settings a little, and go again.

  3. It comes out able to continue any text, along with everything true, false and prejudiced that texts contain. It doesn't behave like an assistant yet.

  4. People teach it that part. They write examples of good answers and compare answers two at a time. For Llama 2, Meta bought Pairwise comparisons of two answers, judged by annotators, collected by Meta to train Llama 2's reward modelMeta's own data only, bought from vendors; one comparison is one annotator choosing the better of two answersSource: Touvron et al., Llama 2: Open Foundation and Fine-Tuned Chat Models (arXiv 2307.09288) (July 18, 2023)Read at the source, from the organization that produced it..

  5. This work is often outsourced. A TIME investigation found that Kenyan workers who sorted violent texts for OpenAI took home Take-home hourly pay of the Kenyan Sama workers who labeled violent text for OpenAI, late 2021One dated case, not annotators' pay in general; OpenAI paid Sama 12.50 dollars an hour, six to nine times what the employees took homeSource: TIME, OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic (January 18, 2023)Read in a press report, not at the source..

Takeaway

The model learned patterns from a huge amount of text. To turn it into an assistant, people wrote examples and rated its answers: there's a lot of human work behind it.

Go further

How much the last stage weighs

Tuning by people takes little computing next to pre-training. For InstructGPT, a close relative of ChatGPT, it took Share of GPT-3's training compute that InstructGPT's human-feedback tuning takes (60 versus 3,640 petaflops/s-days)175-billion-parameter model, 2022; the preferences came from a team of 40 contractorsSource: Ouyang et al., Training language models to follow instructions with human feedback (arXiv 2203.02155) (March 4, 2022)Read at the source, from the organization that produced it. of the compute used to train GPT-3. That number comes from an older generation of models, and this site found no more recent source.

What about Qwen3-4B?

For its small models, Qwen3-4B included, the Qwen team took another route: they learned by imitating the larger models in the family, which had themselves been tuned with human preferences, among other things.

Texts have authors

The books and articles read during training were written by someone. Authors took Anthropic to court over books downloaded from pirate libraries, and the company agreed to pay Settlement of the authors' class action against Anthropic over books downloaded from pirate libraries, about 3,000 dollars per book482,460 books on the list; the settlement compensates pirated copies, not training itselfSource: U.S. District Court for the Northern District of California, Bartz v. Anthropic, Order on Final Approval (3:24-cv-05417-AMO) (July 20, 2026)Read at the source, from the organization that produced it.. The settlement pays for pirated copies. It doesn't say whether training a model on books is allowed.

Sources