Pretrain Vision and Large Language Models in Python End-To-end Techniques for Building and Deploying Foundation Models on AWS
Master the art of training vision and large language models with conceptual fundaments and industry-expert guidance. Learn about AWS services and design patterns, with relevant coding examples Key Features Learn to develop, train, tune, and apply foundation models with optimized end-to-end pipelines...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | eBook |
| Language: | Inglés |
| Published: |
Birmingham :
Packt Publishing, Limited
2023.
|
| Edition: | 1st ed |
| Subjects: | |
| See on Biblioteca Universitat Ramon Llull: | https://discovery.url.edu/permalink/34CSUC_URL/1im36ta/alma991009752735806719 |
Table of Contents:
- Cover
- Title Page
- Copyright and Credits
- Foreword
- Contributors
- Table of Contents
- Preface
- Part 1: Before Pretraining
- Chapter 1: An Introduction to Pretraining Foundation Models
- The art of pretraining and fine-tuning
- The Transformer model architecture and self-attention
- State-of-the-art vision and language models
- Top vision models as of April 2023
- Contrastive pretraining and natural language supervision
- Top language models as of April 2023
- Language technique spotlight - causal modeling and the scaling laws
- Encoders and decoders
- Summary
- References
- Chapter 2: Dataset Preparation: Part One
- Finding a dataset and use case for foundation modeling
- Top pretraining use cases by industry
- Delta - how different is your dataset?
- Use the scaling laws to size your datasets
- Fundamentals - scaling laws of neural language models
- Bias detection and mitigation
- Enhancing your dataset - multilingual, multimodal, and augmentations
- Summary
- References
- Chapter 3: Model Preparation
- Finding your best base model
- Starting with the smallest base model you can
- Trade-off - simplicity versus complexity
- Finding your pretraining loss function
- Pretraining loss functions in vision - ViT and CoCa
- Pretraining loss functions in language - Alexa Teacher Model
- Changing your pretraining loss function
- Solving for your model size
- Practical approaches to solving for your model size
- Not all scaling laws are created equal
- Planning future experiments
- Summary
- References
- Part 2: Configure Your Environment
- Chapter 4: Containers and Accelerators on the Cloud
- What are accelerators and why do they matter?
- Getting ready to use your accelerators
- How to use accelerators on AWS - Amazon SageMaker
- Optimizing accelerator performance
- Hyperparameters.
- Infrastructure optimizations for accelerators on AWS
- Troubleshooting accelerator performance
- Summary
- References
- Chapter 5: Distribution Fundamentals
- Understanding key concepts - data and model parallelism
- What data parallel is all about
- What model parallel is all about
- Combining model and data parallel
- Distributed training on Amazon SageMaker
- Distributed training software
- SM DDP
- SMP library
- Advanced techniques to reduce GPU memory
- Tensor parallelism
- Optimizer state sharding
- Activation checkpointing
- Sharded data parallelism
- Bringing it all home with examples from models today
- Stable Diffusion - data parallelism at scale
- GPT-3 - model and data parallelism at scale
- Summary
- References
- Chapter 6: Dataset Preparation: Part Two, the Data Loader
- Introducing the data loader in Python
- Building and testing your own data loader - a case study from Stable Diffusion
- Creating embeddings - tokenizers and other key steps for smart features
- Optimizing your data pipeline on Amazon SageMaker
- Transforming deep learning datasets at scale on AWS
- Summary
- References
- Part 3: Train Your Model
- Chapter 7: Finding the Right Hyperparameters
- Hyperparameters - batch size, learning rate, and more
- Key hyperparameters in vision and language
- Tuning strategies
- Hyperparameter tuning for foundation models
- Scaling up as a function of world size with SageMaker
- Tuning on a sample of your data and updating based on world size
- Summary
- References
- Chapter 8: Large-Scale Training on SageMaker
- Optimizing your script for SageMaker training
- Importing packages
- Argument parsing
- Top usability features for SageMaker training
- Warm pools for rapid experimentation
- SSM and SSH into training instances
- Track jobs and experiments to replicate results
- Summary.
- References
- Chapter 9: Advanced Training Concepts
- Evaluating and improving throughput
- Calculating model TFLOPS
- Using Flash Attention to speed up your training runs
- Speeding up your jobs with compilation
- Integrating compilation into your PyTorch scripts
- Amazon SageMaker Training Compiler and Neo
- Best practices for compilation
- Running compiled models on Amazon's Trainium and Inferentia custom hardware
- Solving for an optimal training time
- Summary
- References
- Part 4: Evaluate Your Model
- Chapter 10: Fine-Tuning and Evaluating
- Fine-tuning for language, text, and everything in between
- Fine-tuning a language-only model
- Fine-tuning vision-only models
- Fine-tuning vision-language models
- Evaluating foundation models
- Model evaluation metrics for vision
- Model evaluation metrics in language
- Model evaluation metrics in joint vision-language tasks
- Incorporating the human perspective with labeling through SageMaker Ground Truth
- Reinforcement learning from human feedback
- Summary
- References
- Chapter 11: Detecting, Mitigating, and Monitoring Bias
- Detecting bias in ML models
- Detecting bias in large vision and language models
- Mitigating bias in vision and language models
- Bias mitigation in language - counterfactual data augmentation and fair loss functions
- Bias mitigation in vision - reducing correlation dependencies and solving sampling issues
- Monitoring bias in ML models
- Detecting, mitigating, and monitoring bias with SageMaker Clarify
- Summary
- References
- Chapter 12: How to Deploy Your Model
- What is model deployment?
- What is the best way to host my model?
- Model deployment options on AWS with SageMaker
- Why should I shrink my model, and how?
- Model compilation
- Knowledge distillation
- Quantization
- Hosting distributed models on SageMaker.
- Model servers and end-to-end hosting optimizations
- Summary
- References
- Part 5: Deploy Your Model
- Chapter 13: Prompt Engineering
- Prompt engineering - the art of getting more with less
- From few- to zero-shot learning
- Text-to-image prompt engineering tips
- Image-to-image prompt engineering tips
- Upscaling
- Masking
- Prompting for object-to-image with DreamBooth
- Prompting large language models
- Instruction fine-tuning
- Chain-of-thought prompting
- Summarization
- Defending against prompt injections and jailbreaking
- Advanced techniques - prefix and prompt tuning
- Prefix tuning
- Prompt tuning
- Summary
- References
- Chapter 14: MLOps for Vision and Language
- What is MLOps?
- Common MLOps pipelines
- Continuous integration and continuous deployment
- Model monitoring and human-in-the-loop
- MLOps for foundation models
- MLOps for vision
- AWS offerings for MLOps
- A quick introduction to SageMaker Pipelines
- Summary
- References
- Chapter 15: Future Trends in Pretraining Foundation Models
- Techniques for building applications for LLMs
- Building interactive dialogue apps with open-source stacks
- Using RAG to ensure high accuracy in LLM applications
- Is generation the new classification?
- Human-centered design for building applications with LLMs
- Other generative modalities
- AWS offerings in foundation models
- The future of foundation models
- The future of pretraining
- Summary
- References
- Index
- Other Books You May Enjoy.
