Ch 15: MLOps & Model Deployment - Advanced¶
Track: Practitioner | Try code in Playground | Back to chapter overview
Read online or run locally
To run the code interactively, clone the repo and open chapters/chapter-15-mlops-and-model-deployment/notebooks/03_advanced_mlops.ipynb in Jupyter.
Chapter 15: MLOps — Notebook 03 (Advanced MLOps)¶
This notebook covers data and prediction drift (PSI, KS), an Evidently sketch with NumPy fallback, A/B and canary traffic splitting, structured logs and Prometheus metrics, and scaling & cost trade-offs.
What you'll learn¶
| Topic | Section |
|---|---|
| Data drift via PSI and KS tests | §1 |
| Prediction drift and Evidently sketch (with NumPy fallback) | §2 |
| A/B testing and canary traffic splitting | §3 |
| Structured logs and Prometheus metrics | §4 |
| Autoscaling and cost trade-offs | §5 |
| Capstone design: end-to-end MLOps system | §6 |
Time estimate: 2.5 hours
Key concepts¶
- PSI / KS — Catch input-distribution shift before it silently degrades predictions.
- Prediction drift — Watch the output distribution too; sudden shifts often beat input drift to alerting.
- A/B vs canary — A/B compares two models on equal traffic; canary trickles new traffic to the candidate.
- Structured logs — JSON logs with request ID + version + latency are searchable and aggregatable.
- Rollback policy — Define automatic rollback triggers before you need them in an incident.
Run the full notebook for code and outputs.
Generated by Berta AI