Doctoral colloquium - Jozef Kubík (20.3.2023)

Monday 20.3.2023 at 13:10, Lecture room I/9

---

Jozef Kubík:
Active Learning in Large Language Models

Abstract:
In recent years the popularity of creating large language models has been incredibly rising. Most modern LLMs based on Transformers architecture offer great accuracy in many different text-based tasks but are often limited in some areas. For many low or mid-resource languages (such as Slovak), one of the biggest limitations is the amount of annotated data needed for fine-tuning such big model. Our work aims to highlight this problem with a BERT-line of models and suggest a promising method of reducing data for low-resource languages based on the recent developments in the area of Active learning thanks to the novel concept of Epistemic neural networks.

---