SLM series - Cotera: 10 key SLM facts for software engineers (and businesses)
This is a guest post for the Computer Weekly Developer Network written by Tom Firth, co-founder of Cotera.
Cotera is an AI operating system (OS) that unifies support tickets, feedback and product data to turn “millions of interactions” into insights that automate customer experience at scale.
Essentially, Cotera is a customer analytics platform designed to help businesses, particularly customer experience (CX) teams, gain a comprehensive view of customer data across various sources, allowing them to identify trends, proactively address potential issues and make data-driven decisions to improve customer retention and revenue by understanding customer sentiment at scale.
Firth writes in full as follows…
As with all things engineering, the choice between an LLM and an SLM is about tradeoffs. While LLMs operate with billions of parameters and broad knowledge bases, SLMs function with a more focused scope, utilising millions to a few billion parameters. These differences lead to distinct advantages in deployment, efficiency and specificity.
Here are my 10 things you need to know about SLMs.
1. Hybrid Approaches
Combining SLMs and LLMs into integrated systems can be highly effective. A notable study from Microsoft found that using SLMs for routine queries while reserving LLMs for complex tasks reduced computational costs by 40% without compromising performance. In other words, hybrid approaches can give you the best of both worlds, at the cost of a more complex implementation.
2. Intelligent Routing
Intelligent routing systems are central to hybrid deployments. These systems direct queries to the most appropriate model based on complexity and domain specificity. Some advanced implementations even utilise neural networks (generally an SLM) to predict optimal model selection, ensuring efficient resource allocation while maintaining response quality.
3. Dedicated SLMs vs. Single LLM
The choice between multiple specialised SLMs and a single comprehensive LLM presents an architectural decision with significant implications. While a single LLM offers versatility, evidence suggests that dedicated SLMs can achieve superior performance in specialised domains, particularly when tasks can be clearly delineated. The trade-off is that an SLM will require fine-tuning for the specific task at hand. So, while you can get a better result in specialised domains, it takes a lot more work and you can’t iterate as quickly.
4. Training Speed and Data Engineering
The training efficiency of SLMs represents a significant operational advantage. While state-of-the-art LLMs require months of training on specialised hardware, SLMs can be trained or fine-tuned in days or hours. This acceleration in the development cycle enables more frequent model updates and faster adaptation to new data or requirements.
5. Deployment Considerations
SLMs are easy to deploy in a range of contexts – some models can even be deployed on mobile devices. This means that it is tractable for a business to deploy an SLM in a private cloud or on-premises. This is valuable in regulated industries where data privacy concerns necessitate local processing. The ability to deploy SLMs within existing infrastructure has made them increasingly attractive for sensitive applications in healthcare, finance and government sectors.
6. Computational Footprint
Tom Firth, co-founder of Cotera.
The environmental impact of AI systems has become increasingly relevant and SLMs are superior in this domain. The reduced computational requirements of SLMs translate to substantially lower energy consumption in both training and inference phases, aligning with organisational sustainability goals while maintaining practical utility.
7. Limitations and Bias
While SLMs have advantages in specific applications, they are inherently more limited in reasoning capacity and knowledge. The limitations of the models have to be more carefully considered in a developer’s (and therefore an organisation’s) specific domain… and it can take a lot of work to make them effective. LLMs are broadly effective with almost no effort and this advantage alone makes them the right choice for many applications.
8. Domain-Specific Applications
In specialised domains, well-designed SLMs can achieve comparable performance to domain-specific LLMs, often at a fraction of the compute cost. Large domain-specific models like (e.g. BloombergGPT) are indeed better still, but in practice, most organisations will not be able to produce such a model. The practical benefits of SLMs often outweigh the marginal performance gains in many applications.
9. Best Applications
SLMs are strongest in high-throughput, domain-specific applications such as classification, sentiment analysis and specialised text generation. Handling a large volume of similar tasks in a consistent way and with few resources is what they’re good for… and many workloads do look like this.
10. Industry Benefits
SLMs are good in certain verticals because they’re easy to customise to that vertical. As such, they’ve had big impact in a few domains. In healthcare, they enable HIPAA-compliant [a US mandate equivalent to the UK’s data protection act] processing of medical records; in financial services, they facilitate real-time market analysis; and in retail, they power efficient customer service operations.
These are just a few (my first 10) examples!
