Using Deployment Configurations

This tutorial is relevant only for Kubernetes installation of Hydrosphere. Please refer to How to Install Hydrosphere on Kubernetes cluster.
Overview
In this tutorial, you will learn how to configure deployed Applications.
By the end of this tutorial you will know how to:
Train and upload an example model version​
Create a Deployment Configuration​
Create an Application from the uploaded model version with previously created deployment configuration
Examine settings of a Kubernetes cluster
Prerequisites
​Hydrosphere platform installed in Kubernetes cluster​
​Python SDK / CLI​
Upload a Model
In this section, we describe the resources required to create and upload an example model used in further sections. If you have no prior experience with uploading models to the Hydrosphere platform we suggest that you visit the Getting Started Tutorial.
Here are the resources used to train sklearn.ensemble.GradientBoostingClassifier and upload it to the Hydrosphere cluster.
requirements.txt
serving.yaml
train.py
func_main.py
requirements.txt is a list of Python dependencies used during the process of building model image.
1
numpy~=1.18
2
scipy==1.4.1
3
scikit-learn~=0.23
4
pandas==1.3.1
Copied!
serving.yaml is a resource definition that describes how model should be built and uploaded to Hydrosphere platform.
serving.yaml
1
kind: Model
2
name: my-model
3
runtime: hydrosphere/serving-runtime-python-3.7:3.0.0
4
install-command: pip install -r requirements.txt
5
payload:
6
  - src/
7
  - requirements.txt
8
  - model.joblib
9
contract:
10
  name: infer
11
  inputs:
12
    x:
13
      shape: [30]
14
      type: double
15
      profile: numerical
16
  outputs:
17
    y:
18
      shape: scalar
19
      type: int64
20
      profile: numerical
Copied!
train.py is used to generate a model.joblib which is loaded from func_main.py during model serving.
Run python train.py to generate model.joblib
train.py
1
import joblib
2
import pandas as pd
3
from sklearn.datasets import make_blobs
4
from sklearn.ensemble import GradientBoostingClassifier
5
​
6
# initialize data
7
X, y = make_blobs(n_samples=3000, n_features=30)
8
​
9
# create a model
10
model = GradientBoostingClassifier(n_estimators=200)
11
model.fit(X, y)
12
​
13
# Save training data and model
14
pd.DataFrame(X).to_csv("training_data.csv", index=False)
15
joblib.dump(model, "model.joblib")
Copied!
func_main.py is a script which serves requests and produces responses.
func_main.py
1
import joblib
2
import numpy as np
3
​
4
# Load model once
5
model = joblib.load("/model/files/model.joblib")
6
​
7
​
8
def infer(x):
9
    # Make a prediction
10
    y = model.predict(x[np.newaxis])
11
​
12
    # Return the scalar representation of y
13
    return {"y": np.asscalar(y)}
Copied!
Our folder structure should look like this:
1
dep_config_tutorial
2
├── model.joblib
3
├── train.py
4
├── requirements.txt
5
├── serving.yaml
6
└── src
7
    └── func_main.py
Copied!
Do not forget to run python train.py to generate model.joblib!
After we have made sure that all files are placed correctly, we can upload the model to the Hydrosphere platform by running hs apply from the command line.
1
hs apply -f serving.yaml
Copied!
Create a Deployment Configuration
Next, we are going to create and upload an instance of Deployment Configuration to the Hydrosphere platform.
Deployment Configurations describe with which Kubernetes settings Hydrosphere should deploy servables. You can specify Pod Affinity and Tolerations, the number of desired pods in deployment, ResourceRequirements, and Environment Variables for the model container, and HorizontalPodAutoScaler settings.
Created Deployment Configurations can be attached to Servables and Model Variants inside of Application.
Deployment Configurations are immutable and cannot be changed after they've been uploaded to the Hydrosphere platform.
You can create and upload Deployment Configuration to Hydrosphere via YAML Resource definition or via Python SDK.
For this tutorial, we'll create a deployment configuration with 2 initial pods per deployment, HPA, and FOO environment variable with value bar.
YAML Resource Definition
Python SDK
Create the deployment configuration resource definition:
deployment_configuration.yaml
1
kind: DeploymentConfiguration
2
name: my-dep-config
3
deployment:
4
  replicaCount: 2
5
hpa:
6
  minReplicas: 2
7
  maxReplicas: 4
8
  cpuUtilization: 70
9
container:
10
  env:
11
    FOO: bar
Copied!
To upload it to the Hydrosphere platform, run:
1
hs apply -f deployment_configuration.yaml
Copied!
1
from hydrosdk import Cluster, DeploymentConfigurationBuilder
2
​
3
cluster = Cluster("http://localhost")
4
​
5
dep_config_builder = DeploymentConfigurationBuilder("my-dep-config-sdk")
6
dep_config = dep_config_builder \
7
    .with_replicas(replica_count=2) \
8
    .with_env({"FOO":"bar"}) \
9
    .with_hpa(max_replicas=4,
10
             min_replicas=2,
11
             target_cpu_utilization_percentage=70).build(cluster)
Copied!
Create an Application
YAML Resource Definition
Python SDK
Create the application resource definition:
application.yaml
1
kind: Application
2
name: my-app-with-config
3
singular:
4
  model: my-model:1
5
  deployment-config: my-dep-config
Copied!
To upload it to the Hydrosphere platform, run:
1
hs apply -f application.yaml
Copied!
1
from application import ApplicationBuilder, ExecutionStageBuilder
2
from hydrosdk import ModelVersion, Cluster, DeploymentConfiguration
3
​
4
cluster = Cluster('http://localhost')
5
my_model = ModelVersion.find(cluster, "my-model", 1)
6
my_config = DeploymentConfiguration.find(cluster, "my-dep-config")
7
​
8
stage = ExecutionStageBuilder().with_model_variant(model_version=my_model,
9
                                                   weight=100,
10
                                                   deployment_configuration=my_config).build()
11
​
12
app = ApplicationBuilder("my-app-with-config-sdk").with_stage(stage).build(cluster)
Copied!
Examine Kubernetes Settings
Replicas
You can check whether with_replicas was successful by calling kubectl get deployment -A -o wide and checking the READYcolumn.
HPA
To check whether with_hpa was successful you should get a list of all created Horizontal Pod Autoscaler Resources. You can do so by calling kubectl get hpa -A
The output is similar to:
1
NAME                        REFERENCE                                            TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
2
my-model-1-tumbling-star    CrossVersionObjectReference/my-model-1-tumbling-star 20%/70%    2         4         2          1d
Copied!
Environment Variables
To list all environment variables run kubectl exec my-model-1-tumbling-star -it /bin/bash and then execute the printenv command which prints ann system variables.
The output is similar to:
1
MY_MODEL_1_TUMBLING_STAR_SERVICE_PORT_GRPC=9091
2
...
3
FOO=bar
Copied!

A/B Analysis for a Recommendation Model

Train & Deploy Census Income Classification Model

Last modified 5mo ago

Export as PDF

Copy link

Contents

Overview

Prerequisites

Upload a Model

Create a Deployment Configuration

Create an Application

Examine Kubernetes Settings

Replicas

HPA

Environment Variables