FMPI CU researchers speed up AI and make history
Researchers at the Faculty of Mathematics, Physics and Informatics at Comenius University (FMPI CU) Bratislava have achieved a major breakthrough in artificial intelligence research. A team led by Dr Vladimír Boža and doctoral student Vladimír Macko has developed a method that can make large language models (LLMs) significantly faster and less resource-intensive. Their work has attracted international attention and earned prestigious Spotlight recognition at the International Conference on Machine Learning (ICML) in Seoul.
Large language models, including those behind familiar chatbots, have enormous potential. Running them, however, requires vast amounts of computing power and memory. “An LLM is essentially a computation that can be described using a large set of numbers – tens to hundreds of billions of them – which we call weights. Storing these weights is a major challenge,” explains Associate Professor Vladimír Boža of the faculty’s Department of Computer Science.
The most common way to reduce the size of these models, he explains, is to lower the precision of the numbers, a technique known as quantisation. Another approach is pruning: setting unnecessary connections in the network to zero to produce a leaner, “sparse” network. Until now, however, this approach has faced a fundamental obstacle.
“While sparsity looked great in theory, it often failed to deliver in practice on modern graphics processing units (GPUs), because the hardware is designed for large-scale computations with a predictable structure,” explains Vladimír Boža. As a result, even removing half the numbers from a model made little difference to processing times on standard hardware.
The researchers have now overcome this hardware obstacle. Their new method, MACKO-SpMV, combines an efficient way of storing data with algorithms specifically optimised for GPUs. “MACKO-SpMV allows sparse models to run quickly on modern hardware, making them a practical alternative rather than just a theoretical dream,” says Vladimír Boža. At 50 per cent sparsity, the team achieved processing speeds 1.2 to 1.5 times as high while halving memory use: all without the need to buy specialised hardware.
A historic achievement for Slovak AI research
The significance of the breakthrough was recognised when the team’s paper was accepted at the International Conference on Machine Learning (ICML) in Seoul, South Korea. Alongside NeurIPS and ICLR, ICML is one of the world’s three most prestigious AI research conferences. The CU researchers’ paper was selected for a Spotlight presentation, a distinction reserved for roughly the top 3–5 per cent of the thousands of submissions received from around the world.
This marks a major milestone for Slovak science: the first time a research team from a Slovak institution has received Spotlight recognition at ICML. The software library developed by the team is freely available as open-source software, and its practical value has also been recognised by technology giant NVIDIA, which featured it on its official developer blog. The research was supported by grants APVV-24-0045, VEGA-1/0140/2 and VEGA-1/0538/2. The authors also thank the administrators of the Devana and Leonardo high-performance computing clusters for providing computing resources.
Jana Chovancová