- Duration: 4 days
Overview
Demonstrate your capability to automate, monitor, and manage enterprise-grade AI solutions.
This course prepares participants to design, implement, and operate Machine Learning Operations (MLOps) and Generative AI Operations (GenAIOps) solutions on Microsoft Azure.
Participants will learn how to build secure and scalable AI infrastructure, manage the complete lifecycle of traditional machine learning models using Azure Machine Learning, and deploy, evaluate, monitor, and optimize generative AI applications and agents using Microsoft Foundry.
The course provides practical experience with automation, continuous integration and delivery, infrastructure as code, model lifecycle management, AI evaluation, and observability using technologies such as GitHub Actions, Azure CLI, Bicep, MLflow, and Microsoft Foundry.
The course also emphasizes collaboration between data science and DevOps teams to deliver reliable, scalable, secure, and production-ready AI systems aligned with modern MLOps and GenAIOps practices.
Skills Covered
- Design and implement an MLOps infrastructure
- Manage the machine learning model lifecycle
- Implement automated machine learning workflows
- Configure and manage machine learning pipelines
- Implement CI/CD workflows for machine learning solutions
- Automate Azure Machine Learning jobs using GitHub Actions
- Deploy machine learning models through automated workflows
- Design and implement a GenAIOps infrastructure
- Manage prompts and AI agent versions using Git and GitHub
- Design and implement generative AI evaluation workflows
- Automate AI evaluations using Microsoft Foundry and GitHub Actions
- Monitor and troubleshoot generative AI applications
- Implement tracing and observability for AI applications
- Optimize generative AI applications and agent performance
Who Should Attend
This course is designed for:
- Machine Learning Engineers
- AI Engineers
- Data Scientists
- MLOps Engineers
- DevOps Engineers
- Generative AI Developers
- Software Developers
- Cloud Engineers
- AI Solution Architects
- Technical professionals responsible for deploying and operating AI solutions
The course is particularly suitable for professionals who need to automate, deploy, monitor, evaluate, and manage machine learning and generative AI workloads in production environments.
Prerequisites
Before attending this course, participants should have:
- Programming experience with Python or R
- Experience developing and training machine learning models
- Basic understanding of machine learning concepts
- Familiarity with Azure Machine Learning concepts
- Basic understanding of Git and GitHub is recommended
Curriculum
Module 01: Experiment with Azure Machine Learning
Learn how to experiment with machine learning models and evaluate their performance using Azure Machine Learning.
Topics include:
- Azure Machine Learning fundamentals
- Data preprocessing
- Configuring featurization
- Automated Machine Learning experiments
- Model evaluation and comparison
- MLflow for model tracking
- Training and tracking models in notebooks
- Responsible AI dashboard
- Model analysis and responsible AI considerations
- Practical Exercise: Find the Best Classification Model with Azure Machine Learning
- Module Assessment
Module 02: Perform Hyperparameter Tuning with Azure Machine Learning
Learn how to optimize machine learning models by systematically searching for effective hyperparameter combinations.
Topics include:
- Defining a hyperparameter search space
- Configuring sampling methods
- Early termination strategies
- Sweep jobs
- Hyperparameter tuning
- Model optimization
- Practical Exercise: Run a Sweep Job
- Module Assessment
Module 03: Run Pipelines in Azure Machine Learning
Learn how to create and execute reusable machine learning pipelines for repeatable AI workflows.
Topics include:
- Azure Machine Learning components
- Creating reusable components
- Pipeline development
- Pipeline jobs
- Managing pipeline execution
- Reusable machine learning workflows
- Practical Exercise: Run a Pipeline Job
- Module Assessment
Module 04: Trigger Azure Machine Learning Jobs with GitHub Actions
Learn how to integrate Azure Machine Learning workflows with GitHub Actions to automate machine learning processes.
Topics include:
- Understanding the business requirements
- Exploring the solution architecture
- GitHub Actions fundamentals for ML workflows
- Triggering Azure Machine Learning jobs
- Automated model training
- CI/CD integration
- Practical Exercise
- Module Assessment
Module 05: Trigger GitHub Actions with Feature-Based Development
Learn how to implement Git-based development workflows that automatically trigger actions based on feature development and repository changes.
Topics include:
- Understanding the business problem
- Solution architecture
- Feature-based development
- GitHub workflow triggers
- Automated workflow execution
- Integrating development workflows with ML operations
- Practical Exercise
- Module Assessment
Module 06: Work with Environments in GitHub Actions
Learn how to configure and manage deployment environments within GitHub Actions.
Topics include:
- Understanding environment requirements
- GitHub Actions environments
- Environment configuration
- Environment-specific workflows
- Managing deployment environments
- Supporting secure and controlled deployments
- Practical Exercise
- Module Assessment
Module 07: Deploy a Model with GitHub Actions
Learn how to automate machine learning model deployment using GitHub Actions.
Topics include:
- Understanding the deployment requirements
- Solution architecture
- Model deployment workflows
- Automated deployment
- CI/CD for machine learning models
- Deployment automation
- Practical Exercise
- Module Assessment
Module 08: Plan and Prepare a GenAIOps Solution
Learn how to plan and establish an operational framework for generative AI applications.
Topics include:
- Introduction to GenAIOps
- GenAIOps use cases
- Selecting appropriate generative AI models
- Language model application lifecycle
- Generative AI development workflows
- GenAIOps tools and frameworks
- Planning operational AI environments
- Practical Exercise: Compare Language Models from the Model Catalog
- Module Assessment
Module 09: Manage Prompts for Agents in Microsoft Foundry with GitHub
Learn how to manage, version, test, and safely deploy prompts used by AI agents.
Topics include:
- Prompt version control
- Microsoft Foundry agents
- Prompt versioning
- Organizing prompts in GitHub repositories
- Git-based prompt management
- Safe prompt deployment workflows
- Agent version management
- Practical Exercise: Develop Prompt and Agent Versions
- Knowledge Check
Module 10: Evaluate and Optimize AI Agents Through Structured Experiments
Learn how to systematically evaluate and improve AI agent performance through structured experimentation.
Topics include:
- Designing evaluation experiments
- AI agent optimization
- Git-based experimentation workflows
- Evaluation rubrics
- Consistent AI quality scoring
- Comparing different agent versions
- Performance optimization
- Practical Exercise: Evaluate and Compare AI Agent Versions
- Knowledge Check
Module 11: Automate AI Evaluations with Microsoft Foundry and GitHub Actions
Learn how to automate the evaluation of generative AI applications and agents.
Topics include:
- Importance of automated AI evaluation
- Aligning evaluators with human evaluation criteria
- Creating evaluation datasets
- Batch evaluations
- Python-based evaluation workflows
- Integrating evaluations with GitHub Actions
- Automated quality assurance
- Continuous AI evaluation
- Practical Exercise: Set Up Automated Evaluations
- Knowledge Check
Module 12: Monitor Your Generative AI Application
Learn how to monitor generative AI applications and use operational metrics to understand application performance and behavior.
Topics include:
- Importance of generative AI monitoring
- Key AI application metrics
- Microsoft Azure monitoring capabilities
- Integrating monitoring into AI applications
- Application performance monitoring
- Interpreting monitoring results
- Identifying operational issues
- Practical Exercise: Enable Monitoring for a Generative AI Application
- Knowledge Check
Module 13: Analyze and Debug Your Generative AI App with Tracing
Learn how to use tracing to understand, troubleshoot, and optimize complex generative AI application workflows.
Topics include:
- Importance of tracing
- Identifying components and operations to trace
- Implementing tracing in generative AI applications
- Distributed and workflow tracing
- Debugging complex AI workflows
- Advanced tracing patterns
- Analyzing trace data
- Using trace information for optimization and troubleshooting
- Practical Exercise: Enable Tracing for a Generative AI Application
- Knowledge Check
Course Outcome
Upon successful completion of this course, participants will be able to design, automate, deploy, evaluate, monitor, and optimize machine learning and generative AI solutions on Microsoft Azure.
Participants will gain practical skills in MLOps, GenAIOps, CI/CD automation, model lifecycle management, AI evaluation, observability, tracing, and production AI operations, enabling them to support reliable and scalable enterprise AI solutions.