AI FRONTIERS: LARGE LANGUAGE MODELS
Instructional goals
Provide an in-depth understanding of Large Language Models (LLM) and Generative AI.
Develop skills necessary to implement and use LLM in various application contexts.
Explore the ethical implications and social impact of using LLM and Generative AI.
Intended learning outcomes
Understand the fundamental principles of LLM and Generative AI.
Ability to implement language models using machine learning libraries.
Analyze and evaluate the performance of LLM in various tasks.
Identify and discuss the ethical considerations related to the use of LLM.
Course Contents
Introduction to LLM and Generative AI (4 hours)
Language Model Architectures (8 hours)
Recurrent Neural Networks (RNN)
Transformers and BERT
Training LLM (8 hours)
Datasets and preprocessing
Optimization techniques
Applications of LLM (8 hours)
Text generation
Machine translation
Chatbots and virtual assistants
Tools and Frameworks (4 hours)
TensorFlow and PyTorch
Specific NLP libraries
Evaluation and Fine-tuning (4 hours)
Evaluation metrics
Fine-tuning techniques
Ethics and Social Impact (4 hours)
Bias in language models
Social impact and regulation
Practical Project (8 hours)
Implementation of an LLM
Project presentation
Reference Books
"Deep Learning" by Ian Goodfellow, Yoshua Bengio, Aaron Courville
"Natural Language Processing with PyTorch" by Delip Rao, Brian McMahan
Articles and online resources on LLM and Generative AI
Teaching Methods
Lectures
Practical lab exercises
Group discussions and case study analysis
Practical projects and presentations
Assessment Method
Oral presentations
Active participation in classes and discussions
Thesis assignment criteria
Ability to apply the knowledge acquired during the course
Clarity of exposition and argumentation skills
Week 1
Introduction to LLM and Generative AI
Reading: Chapter 1 of "Deep Learning"
Week 2
Language Model Architectures: RNN
Week 3
Language Model Architectures: Transformers and BERT
Reading: "Attention is All You Need" paper
Week 4
Training LLM: Datasets and preprocessing
Week 5
Training LLM: Optimization techniques
Week 6
Applications of LLM: Text generation and prompting
Week 7
Applications of LLM: Machine translation and other language tasks
Week 8
Applications of LLM: Chatbots and virtual assistants and copilots
Week 9
Tools and Frameworks: TensorFlow and PyTorch
Week 10
Tools and Frameworks: Specific NLP libraries
Week 11
Evaluation and Fine-tuning: Evaluation metrics
Week 12
Evaluation and Fine-tuning: Fine-tuning techniques