Back to blog

Pharo LLMs Beat Giant Models

Based on research by Kilian Kier, Alessandro Giagnorio, Omar AbedelKader, Oleksandr Zaitsev, Robert Peharz

Most developers rely on large language models to autocomplete their code, but this technology largely ignores programming languages with smaller communities. If you code in Pharo, a Smalltalk-inspired language, you are stuck with primitive single-token suggestions while mainstream languages enjoy sophisticated AI assistance. This disparity leaves entire developer communities behind, struggling with tools that fail to understand the nuances of their preferred syntax.

Researchers have now demonstrated how to bridge this gap by teaching LLMs to handle low-resource languages. They built an end-to-end pipeline that curates Pharo-specific data, then continues pre-training and fine-tuning open code models. To measure success, they created new benchmarks testing whether models can learn Pharo syntax and accurately complete code snippets from real-world GitHub repositories. This approach moves beyond generic training, focusing specifically on the unique requirements of a niche programming environment.

The results challenge the assumption that only massive models can provide high-quality assistance. The specialized Pharo models substantially outperformed their original base versions and even exceeded the accuracy of significantly larger general-purpose code LLMs. Crucially, these optimized models are small enough to run in real-time within an integrated development environment. This proves that targeted fine-tuning can deliver powerful, immediate coding support without requiring immense computational resources.

The study confirms that bringing strong LLM-based code completion to low-resource languages is not just feasible but highly effective. By focusing on specific data and targeted training, developers can enjoy AI assistance that rivals mainstream tools. This breakthrough offers a clear path for expanding intelligent coding support to diverse programming communities that have been overlooked until now.

Source: arXiv:2607.04939

This post was generated by staik AI based on the academic publication above.