How DeepSeek has changed artificial intelligence and what it means for Europe

March 20, 2025--By mid-2024, artificial intelligence large language models (LLMs) were running into diminishing returns to scale in training data and computational capacity. LLM training began to shift away from costly pre-training to cheaper fine-tuning and allowing LLMs to 'reason' for longer before replying to questions.

Fine-tuning uses chain-of-thought (CoT) training data that includes questions and the logical steps to reach correct answers. This increases the efficiency of learning for smaller AI models, such as DeepSeek. CoT data can be extracted from large 'teacher' LLMs to train small 'student, models.

These changes shift the cost structure of AI models from high pre-training costs to lower fine-tuning costs for model developers and more inference costs for users.

view more

Source: bruegel.org