Doktorandské kolokvium KAI - Jozef Kubík (20.3.2023)
v pondelok 20.3.2023 o 13:10 hod. v miestnosti I/9
Od: Damas Gruska
Prednášajúci: Jozef Kubík
Názov: Active Learning in Large Language Models
Termín: 20.3.2023, 13:10 hod., I/9
In recent years the popularity of creating large language models has been incredibly rising. Most modern LLMs based on Transformers architecture offer great accuracy in many different text-based tasks but are often limited in some areas. For many low or mid-resource languages (such as Slovak), one of the biggest limitations is the amount of annotated data needed for fine-tuning such big model. Our work aims to highlight this problem with a BERT-line of models and suggest a promising method of reducing data for low-resource languages based on the recent developments in the area of Active learning thanks to the novel concept of Epistemic neural networks.