AI FRONTIERS: LARGE LANGUAGE MODELS

AI FRONTIERS: LARGE LANGUAGE MODELS

Simone Di Somma

Instructional goals

Provide an in-depth understanding of Large Language Models (LLM) and Generative AI. Develop skills necessary to implement and use LLM in various application contexts. Explore the ethical implications and social impact of using LLM and Generative AI.

Intended learning outcomes

Understand the fundamental principles of LLM and Generative AI. Ability to implement language models using machine learning libraries. Analyze and evaluate the performance of LLM in various tasks. Identify and discuss the ethical considerations related to the use of LLM.

Course Contents

Introduction to LLM and Generative AI (4 hours) Language Model Architectures (8 hours) Recurrent Neural Networks (RNN) Transformers and BERT Training LLM (8 hours) Datasets and preprocessing Optimization techniques Applications of LLM (8 hours) Text generation Machine translation Chatbots and virtual assistants Tools and Frameworks (4 hours) TensorFlow and PyTorch Specific NLP libraries Evaluation and Fine-tuning (4 hours) Evaluation metrics Fine-tuning techniques Ethics and Social Impact (4 hours) Bias in language models Social impact and regulation Practical Project (8 hours) Implementation of an LLM Project presentation

Reference Books

"Deep Learning" by Ian Goodfellow, Yoshua Bengio, Aaron Courville "Natural Language Processing with PyTorch" by Delip Rao, Brian McMahan Articles and online resources on LLM and Generative AI

Teaching Methods

Lectures Practical lab exercises Group discussions and case study analysis Practical projects and presentations

Assessment Method

Oral presentations Active participation in classes and discussions

Thesis assignment criteria

Ability to apply the knowledge acquired during the course Clarity of exposition and argumentation skills

Week 1

Introduction to LLM and Generative AI Reading: Chapter 1 of "Deep Learning"

Week 2

Language Model Architectures: RNN

Week 3

Language Model Architectures: Transformers and BERT Reading: "Attention is All You Need" paper

Week 4

Training LLM: Datasets and preprocessing

Week 5

Training LLM: Optimization techniques

Week 6

Applications of LLM: Text generation and prompting

Week 7

Applications of LLM: Machine translation and other language tasks

Week 8

Applications of LLM: Chatbots and virtual assistants and copilots

Week 9

Tools and Frameworks: TensorFlow and PyTorch

Week 10

Tools and Frameworks: Specific NLP libraries

Week 11

Evaluation and Fine-tuning: Evaluation metrics

Week 12

Evaluation and Fine-tuning: Fine-tuning techniques