AI60213 - Foundation of Large Language Models

Language being a complex and intricate system of human expression have put forward a significant challenge to AI for its understanding and generation. Language models are the foundation of language understanding and task of language modelling has been evolved from statistical to neural models. In recent times, pre-trained language models (PLMs), founded on the Transformer architecture and trained on exceedingly large amount of data, has shown success in solving majority of the NLP tasks. The performance in various tasks has been observed to improve with the scale of the models. Consequently, several Large Language Models (LLMs) have been developed that not only can solve specific tasks but also has been observed to exhibit emergent behaviour. The LLMs have been adopted in other data modalities like vision, speech or multi-modal. Very recently, LLMs are being used to develop autonomous agents that can solve complex tasks. This course aims at providing foundational knowledge on key technologies for developing, leveraging and augmenting LLMs.