Language Models.
Testing
the Logic.
One riddle. Three Qwen models.
Comparing outputs to uncover what real reasoning looks like.
7B
Qwen2.5
Instruction-tuned
9B
Qwen3.5
Thinking mode
72B
Qwen2.5
Instruction-tuned
B = billion parameters. Largest shown: 72B = 100%.
Size comparison, not an accuracy score.
Sally has 3 brothers.
Each brother has 2 sisters.
How many sisters does Sally have? Explain step by step.
Multiplying 3 × 2 = 6 counts the same sisters more than once. Sally is one of the two sisters.
Qwen2.5-7B
Instruct
FAILEDTranslated & shortened · Russian run
- Sally has 3 brothers.
- Each brother has 2 sisters.
- We add them:
2 + 2 + 2 = 6
Answer: Sally has 6 sisters.
The same people,
counted three times.
One saved response, not an overall model score.
Qwen2.5-72B
Instruct
PASSEDTranslated & shortened · Russian run
“One sister is Sally herself. The other is shared by all three brothers.”
2 sisters − Sally= 1 sister
Answer: Sally has 1 sister.
Family check: 3 boys + 2 girls = 5 children.
Qwen3.5-9B Thinking
PASSEDTranslated & shortened · Russian run
“There are two sisters in the family. Sally is one of them. So she has one other sister.”
Answer: Sally has 1 sister.
9B → 1 sister Same answer as 72B. 8× fewer parameters.
Correct final answer. Its explanation first counted only 4 children; the family has 5.
7B–9B
Try smaller models first, including reasoning-enabled options.
72B
Scale up when an accuracy gain justifies the extra cost.
“Choose the model for the reasoning depth you need, not just the speed.”