What are Small Language Models?

We hear a lot about Large Language Models (LLMs) such as OpenAI’s GPT (Generative Pre-trained Transformer), Google’s Gemini and Anthropic’s Claude.

Small Language Models (SLMs) are much less well known, but include the Phi family of SLMs from Microsoft, Mistral Small, and Llama from Meta.

The “large” or “small” in these models refers to both the number of parameters in the models, and the amount of data used for training. SLMs have both a smaller number of parameters and a smaller training data set when compared to LLMs.

Admittedly, the line between LLM and SLM is blurry. There is no well defined distinction between them, but below I cover some of the common characteristics.

(more…)

Tags: AI, artificialintelligence, largelanguagemodel, llm, slm, smalllanguagemodel