Faculty of Mathematics, Physics
and Informatics
Comenius University Bratislava

Doctoral colloquium - Jozef Kubík (20.3.2023)

Monday 20.3.2023 at 13:10, Lecture room I/9

16. 03. 2023 11.00 hod.
By: Damas Gruska

Jozef Kubík:
Active Learning in Large Language Models

In recent years the popularity of creating large language models has been incredibly rising. Most modern LLMs based on Transformers architecture offer great accuracy in many different text-based tasks but are often limited in some areas. For many low or mid-resource languages (such as Slovak), one of the biggest limitations is the amount of annotated data needed for fine-tuning such big model. Our work aims to highlight this problem with a BERT-line of models and suggest a promising method of reducing data for low-resource languages based on the recent developments in the area of Active learning thanks to the novel concept of Epistemic neural networks.