Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Performance Fundamentals and Key Metrics
- Analysis of latency, throughput, power consumption, and resource utilization
- Distinguishing between system-level and model-level bottlenecks
- Profiling strategies for inference versus training workloads
Profiling on Huawei Ascend
- Utilizing CANN Profiler and MindInsight
- Diagnosing kernels and operators
- Managing offload patterns and memory mapping
Profiling on Biren GPU
- Monitoring performance via Biren SDK features
- Implementing kernel fusion, memory alignment, and execution queue management
- Profiling with awareness of power and thermal constraints
Profiling on Cambricon MLU
- Leveraging BANGPy and Neuware performance utilities
- Interpreting logs and gaining kernel-level visibility
- Integrating MLU profilers with deployment frameworks
Graph and Model-Level Optimizations
- Strategies for graph pruning and quantization
- Restructuring computational graphs through operator fusion
- Standardizing input sizes and tuning batch parameters
Memory and Kernel Optimization
- Optimizing memory layout and data reuse
- Managing buffers efficiently across different chipsets
- Applying platform-specific kernel tuning techniques
Cross-Platform Best Practices
- Achieving performance portability through abstraction strategies
- Developing shared tuning pipelines for multi-chip environments
- Case study: Optimizing an object detection model across Ascend, Biren, and MLU
Conclusion and Future Directions
Requirements
- Practical experience with AI model training or deployment pipelines
- Familiarity with GPU/MLU compute fundamentals and model optimization techniques
- Basic knowledge of performance profiling metrics and associated tools
Target Audience
- Performance Engineers
- Machine Learning Infrastructure Teams
- AI System Architects
