Hydrosphere.io
GithubPython SDKContact UsSlack Community
2.4.2 Release
2.4.2 Release
  • Hydrosphere
  • 🌊About Hydrosphere
    • Overview
    • Concepts
    • Platform Architecture
      • Serving
      • Monitoring
      • Interpretability
    • Key Features
      • Model Registry
      • Inference Pipelines
      • A/B Model Deployments
      • Traffic Shadowing
      • Language-Agnostic
      • Automatic Outlier Detection
      • Data Drift Report
      • Monitoring Dashboard
      • Alerts
      • Prediction Explanation
      • Data Projection
      • Kubeflow Components
      • AWS Sagemaker
  • πŸ„Quickstart
    • Installation
      • CLI
      • Python SDK
    • Getting Started
    • Tutorials
      • A/B Analysis for a Recommendation Model
      • Using Deployment Configurations
      • Train & Deploy Census Income Classification Model
      • Monitoring Anomalies with a Custom Metric
      • Monitoring External Models
    • How-To
      • Invoke applications
      • Write definitions
      • Develop runtimes
      • Use private pip repositories
  • πŸ’§Resources
    • Troubleshooting
    • Reference
      • Runtimes
    • Contribution
      • Contributing Pull Requests
Powered by GitBook
On this page

Was this helpful?

Export as PDF
  1. About Hydrosphere
  2. Key Features

A/B Model Deployments

PreviousInference PipelinesNextTraffic Shadowing

Last updated 4 years ago

Was this helpful?

Hydrosphere allows you to A/B test your ML models in production.

A/B testing is a great way of measuring how well your models perform or which of your model versions is more effective and taking data-driven decisions upon this knowledge.

Production ML applications always have specific goals, for example driving as many users as possible to perform some action. To achieve these goals, it’s necessary to run online experiments and compare model versions using metrics in order to measure your progress against them. This approach allows to track whether your development efforts lead to desired outcomes.

To perform a basic A/B experiment on an application consisting of 2 variants of a model, you need to train and upload both versions to Hydrosphere, create an application with a single execution stage from them, invoke it by simulating production data flow, then analyze production data using metrics of your choice.

Learn how to set up an A/B application:

🌊
A/B Analysis for a Recommendation Model