Book Review: Domain Specific Small Language Models

Artificial Intelligence (AI) powered applications are changing the way we consume and use information. Retrieval Augmented Generation (RAG) systems are becoming ubiquitous with AI generated answers to our questions replacing Google / Bing searches returning a list of links to matching documents. Programmers and non-programmers alike are using Agents backed by Large Language Models (LLM) to write code, create schedules, and a host of other things that we had to do by hand just a few years ago.

Although AI is now everywhere, the intelligence powering them come from large general-purpose LLMs that are run by only a few large companies like OpenAI and Anthropic. As AI gets more integrated into our lives, it will become harder for us to do without it. There is a risk of price gouging (aka a more realistic pricing model) as that happens, as Edward Tufte has noted.

Even with current pricing, LLM pipelines work out to be significantly more expensive and slower than their (albeit less powerful) pre-AI incarnations. A good place to look to lower costs and latency may be to consider whether all the functions that are using LLMs in a pipeline really need the power of these models, or if they can be achieved with (smaller and potentially faster) Domain Specific Small Language Models (SLM). I wanted to educate myself about this, so I picked up Domain Specific Small Language Models to learn more about SLMs. Here is my review.

The book is organized in three parts. Part 1 is basic background material, Part 2 covers the information to get you started with fine-tuning and deploying domain-specific SLMs in your pipelines, Part 3 covers some case studies, and Part 4 covers advanced topics mostly covering late-breaking breakthroughs and tooling popular in real-world pipelines.

Part 1 covers a brief history of LLMs, the Transformer architecture that they use, the contributions of the open source community which led to the development of SLMs, the case for SLMs, etc. If you are looking to learn about SLMs, it is very likely that you have a Machine Learning (ML) background, so you may be familiar with lot of this material already. However, it can be a useful refresher that can round out possible gaps in your knowledge.

Part 2 covers fine-tuning SLMs to adapt them to specific domains. The focus is on fine-tuning generative models (either encoder-decoder or decoder-only Transformer models), which tend to be larger and therefore require specialized training strategies such as LoRA (Low Rank Adaptation). It also discusses strategies to deploy SLMs at scale for inference. Model quantization is a big part of the deployment strategy for inference, and that gets its own chapter. In addition, there is a chapter on ONNX (Open Neural Network eXchange), which is an open format for models such that they can portably run on any ONNX supported runtime (spanning programming language environments).

Part 3 covers some real-world use cases of SLMs, such as Code Generation using fine-tuned SLMs such as StarCoder and CodeGen, and generating chemical and protein structures using ProtGPT2, AntibodyGPT, and CrystaLLM.

Part 4 covers additional advanced quantization techniques (FlexGen, SmoothQuant, BitNet), profiling using the ONNX Runtime (ORT) platform, and deployment platforms such as vLLM and Ollama. It also covers the integration of SLMs with RAG and Agentic pipelines, including a discussion of components to support Agentic pipelines such as GraphRAG, Tools and Memory. Finally, it covers test time compute, which is typically used for inference by reasoning engines, as well as adapting SLMs for reasoning using GRPO (Group Relative Policy Optimization).

There are code examples throughout the book, mostly based on the HuggingFace API. This makes sense, since much of SLM development happens using their API and is hosted on their repositories.

Overall, I think the book can jumpstart your journey into SLMs, but the field is moving too fast for one book to cover. What you will get out of the book are solid foundations that will help you navigate more recent developments in the field and decide if it is something that makes sense to explore for your use case.