Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Introduction to Multimodal LLMs in Vertex AI
- Overview of multimodal capabilities available in Vertex AI
- Exploration of Gemini models and the modalities they support
- Enterprise and research use cases
Setting Up the Development Environment
- Configuration of Vertex AI for multimodal workflows
- Managing datasets across different modalities
- Hands-on lab: environment setup and dataset preparation
Long Context Windows and Advanced Reasoning
- Concepts behind long-context workflows
- Applications in planning and decision-making scenarios
- Hands-on lab: implementing long-context analysis
Cross-Modal Workflow Design
- Integrating text, audio, and image analysis
- Chaining multimodal steps within pipelines
- Hands-on lab: designing a multimodal pipeline
Working with Gemini API Parameters
- Configuring multimodal inputs and outputs
- Optimizing inference speed and efficiency
- Hands-on lab: tuning Gemini API parameters
Advanced Applications and Integrations
- Building interactive multimodal agents and assistants
- Integrating external APIs and tools
- Hands-on lab: developing a multimodal application
Evaluation and Iteration
- Methods for testing multimodal performance
- Metrics for accuracy, alignment, and drift
- Hands-on lab: evaluating multimodal workflows
Summary and Next Steps
Requirements
- Strong proficiency in Python programming
- Experience in developing machine learning models
- Familiarity with handling multimodal data, including text, audio, and images
Audience
- AI researchers
- Advanced developers
- ML scientists
