Doktorandské kolokvium KAI - Jozef Kubík (20.3.2023)

v pondelok 20.3.2023 o 13:10 hod. v miestnosti I/9

---

Prednášajúci: Jozef Kubík 

Názov: Active Learning in Large Language Models

Termín: 20.3.2023, 13:10 hod., I/9

Abstrakt:
In recent years the popularity of creating large language models has been incredibly rising. Most modern LLMs based on Transformers architecture offer great accuracy in many different text-based tasks but are often limited in some areas. For many low or mid-resource languages (such as Slovak), one of the biggest limitations is the amount of annotated data needed for fine-tuning such big model. Our work aims to highlight this problem with a BERT-line of models and suggest a promising method of reducing data for low-resource languages based on the recent developments in the area of Active learning thanks to the novel concept of Epistemic neural networks.

---