SLM series - Qt: Practical code experiences from the command line
This is a guest post for the Computer Weekly Developer Network written by Peter Schneider in his role as senior product manager at Qt Group.
Schneider writes as follows…
On the question of whether SLMs and LLMs be combined and used in union and in concert – yes.
By using both SLMs and LLMs for software development assistance, software developers are able to maximise productivity.
Based on our experience, royalty-free SLMs such as CodeLlama 7B or DeepSeek Coder v2 are very good at QML (Qt Modelling Language) code completion and provide low inference times at essentially no cost. For example, the locally running DeepSeek Coder V2 Lite SLM and the cloud-based GPT4o LLM score the same QML code completion result of 62% successful task execution (consisting of 100 Fill-In-the-Middle tasks).
General purpose LLMs such as Claude 3.5 Sonnet or DeepSeek V3 are great at prompt-based expert assistance for more complex tasks (“fix this code”, “explain this code”, “document this code”, or “create a Qt unit test”). Using a mixture of SLMs and LLMs lead to an ideal user experience for coding assistance.
Intelligent routing
Intelligent routing between language models depending on the task and programming language is essential not only to maximise the quality of the content (code, code documentation, and explanations) but also to reduce costs and increase the privacy of your own code.
Your coding assistant should be able to route code completion requests for your primary coding language, in our case that is QML, to a model which has been fine-tuned with additional knowledge for higher-quality responses. For all other languages a Qt application developer needs, typically this is C++ or Python, they can use a base model which hasn’t been fine-tuned but is strong at those languages.
Based on our fine-tuning research, base models have a habit of forgetting some of its original coding skills when fine-tuning for a dedicated task such as writing QML code. Therefore, routing of code completion requests to different models based on the programming language is essential for maximising the quality of output.
Dedicated SLMs
So then, is a selection of dedicated SLMs better than one LLM?
Again, yes.
As mentioned above SLMs can be as good or even better than LLMs for certain tasks such as QML Fill-In-the-Middle code completion at a much, much lower cost. For this essential task, LLMs are not necessarily required or even desirable. Noteworthy is that a fine-tuned SLMs such as the CodeLlama 13B-QML might is better in some tasks such as code completion.
Qt’s Schneider: SLM can be great at code completion, which for many of our developers is THE most important task for a coding assistant.
After fine-tuning CodeLlama 13B with over 4000 QML code snippets, the fine-tuned model is able to code 79% of QML coding tasks successfully while the highest scoring LLM Claude 3.5 Sonnet “only” achieves 66% successful code completion.
By nature of their size, SLMs faster to train, so for AI data engineering at companies that do not have millions of dollars and a nuclear power plant at their disposal, model fine-tuning needs to be at a reasonable cost. By selecting to fine-tune SLMs such as CodeLlama 7B or 13B with a cost efficient Parameter Efficient Fine-Tuning (PEFT) method, the compute expenses of delivering superior coding assistance performance for dedicated tasks remain commercially feasible.
Whether SLMs are always deployed locally depends a bit on the interpretation of what is really a small model. We are supporting for example both the CodeLlama 7B and the 13B model. It is more typical to run the 7B model locally on your PC while the 13B model with a slightly better QML coding performance can be deployed also in a private cloud to reduce the hardware requirements on the developer’s computer. In either case, private and competitive information remains in these deployment models secured.
The ‘one’ thing for developers
In terms of what limitations SLMs exhibit and whether they harbour the same propensity for bias as large-scale AI technologies? That depends on the task. SLM can be great at tasks like code completion, which for many of our developers is THE most important task for a coding assistant.
Based on our research, solving complex tasks such as generating unit test cases which can involve a prompt consisting of over 50 instructions does not lead to satisfactory outcomes with today’s SLMs.