Course Outline
Preparation of Machine Learning Models for Deployment
- Packaging models using Docker
- Exporting models from TensorFlow and PyTorch
- Versioning and storage considerations
Model Serving on Kubernetes
- Overview of inference servers
- Deploying TensorFlow Serving and TorchServe
- Establishing model endpoints
Inference Optimization Techniques
- Batching strategies
- Managing concurrent request handling
- Tuning latency and throughput
Autoscaling ML Workloads
- Horizontal Pod Autoscaler (HPA)
- Vertical Pod Autoscaler (VPA)
- Kubernetes Event-Driven Autoscaling (KEDA)
GPU Provisioning and Resource Management
- Configuring GPU nodes
- Overview of the NVIDIA device plugin
- Setting resource requests and limits for ML workloads
Model Rollout and Release Strategies
- Blue/green deployments
- Canary rollout patterns
- A/B testing for model evaluation
Monitoring and Observability for Production ML
- Key metrics for inference workloads
- Logging and tracing practices
- Dashboard creation and alerting configurations
Security and Reliability Considerations
- Securing model endpoints
- Implementing network policies and access control
- Ensuring high availability
Summary and Next Steps
Requirements
- An understanding of containerized application workflows
- Experience with Python-based machine learning models
- Familiarity with Kubernetes fundamentals
Target Audience
- ML engineers
- DevOps engineers
- Platform engineering teams
Testimonials (4)
basic understanding of container/kubernetes and how they interact features of the openshift plattform
Eric Scholze - NOW IT GmbH
Course - Introduction to Containers, Kubernetes & OpenShift
About the microservices and how to maintenance kubernetes
Yufri Isnaini Rochmat Maulana - Bank Indonesia
Course - Advanced Platform Engineering: Scaling with Microservices and Kubernetes
How trainer deliver knowledge so effectively
Vu Thoai Le - Reply Polska sp. z o. o.
Course - Certified Kubernetes Administrator (CKA) - exam preparation
The knowledge and exchanges with Augustin
