Write definitions
​Resource definitions describe Hydrosphere entities.
An entity could be your model, application, or deployment configuration. Each definition is represented by a .yaml file.

Base definition

Every definition must include the following fields:
  • kind: defines the type of a resource
  • name: defines the name of a resource
The only valid options for kind are:
  • Model
  • Application
  • DeploymentConfiguration

kind: Model

A model definition must contain the following fields:
  • runtime: a string defining the runtime Docker image that will be used to run a model. You can learn more about runtimes here.
  • contract: an object defining the inputs and outputs of a model.
A model definition can contain the following fields:
  • payload: a list of files that should be added to the container.
  • install-command: a string defining a command that should be executed during the container build.
  • training-data: a string defining a path to the file that will be uploaded to Hydrosphere and used as a training data reference. It can be either a local file or a URI to an S3 object. At the moment we only support .csv files.
  • metadata: an object defining additional user metadata that will be displayed on the Hydrosphere UI.
The example below shows how a model can be defined on the top level.
serving.yaml
1
kind: "Model"
2
name: "sample_model"
3
training-data: "s3://bucket/train.csv" | "/temp/file.csv"
4
runtime: "hydrosphere/serving-runtime-python-3.7:3.0.0"
5
install-command: "sudo apt install jq && pip install -r requirements.txt"
6
payload:
7
- "./requirements.txt"
8
contract:
9
...
10
metadata:
11
...
Copied!

Contract object

contract object must contain the following fields:
  • inputs: an object, defining all inputs of a model
  • outputs: an object, defining all outputs of a model
contract object can contain the following fields:
  • name: a string defining the signature of the model that should be used to process requests

Field object

field object must contain the following fields:
  • shape: either "scalar" or a list of integers, defining the shape of your data. If a shape is defined as a list of integers, it can have -1 value at the very beginning of the list, indicating that this field has an arbitrary number of "entities". -1 cannot be put anywhere aside from the beginning of the list.
  • type: a string defining the type of data.
field object can contain the following fields:
  • profile: a string, defining the profile type of your data.
The only valid options for type are:
  • bool β€” Boolean
  • string β€” String in bytes
  • half β€” 16-bit half-precision floating-point
  • float16 β€” 16-bit half-precision floating-point
  • float32 β€” 32-bit single-precision floating-point
  • double β€” 64-bit double-precision floating-point
  • float64 β€” 64-bit double-precision floating-point
  • uint8 β€” 8-bit unsigned integer
  • uint16 β€” 16-bit unsigned integer
  • uint32 β€” 32-bit unsigned integer
  • uint64 β€” 64-bit unsigned integer
  • int8 β€” 8-bit signed integer
  • int16 β€” 16-bit signed integer
  • int32 β€” 32-bit signed integer
  • int64 β€” 64-bit signed integer
  • qint8 β€” Quantized 8-bit signed integer
  • quint8 β€” Quantized 8-bit unsigned integer
  • qint16 β€” Quantized 16-bit signed integer
  • quint16 β€” Quantized 16-bit unsigned integer
  • complex64 β€” 64-bit single-precision complex
  • complex128 β€” 128-bit double-precision complex
The only valid options for profile are:
  • text β€” monitoring such fields will be done with text-oriented algorithms.
  • image β€” monitoring such fields will be done with image-oriented algorithms.
  • numerical β€” monitoring such fields will be done with numerical-oriented algorithms.
  • categorical β€” monitoring such fields will be done with categorical-oriented algorithms.
The example below shows how a contract can be defined on the top level.
1
name: "infer"
2
inputs:
3
input_field_1:
4
shape: [-1, 1]
5
type: string
6
profile: text
7
input_field_2:
8
shape: [200, 200]
9
type: int32
10
profile: categorical
11
outputs:
12
output_field_1:
13
shape: scalar
14
type: int32
15
profile: numerical
Copied!

Metadata object

metadata object can represent any arbitrary information specified by the user. The structure of the object is not strictly defined. The only constraint is that the object must have a key-value structure, where a value can only be of a simple data type (string, number, boolean).
The example below shows, how metadata can be defined.
1
metadata:
2
experiment: "demo"
3
environment: "kubernetes"
Copied!
The example below shows a complete definition of a sample model.
1
kind: "Model"
2
name: "sample_model"
3
training-data: "s3://bucket/train.csv" | "/temp/file.csv"
4
runtime: "hydrosphere/serving-runtime-python-3.7:3.0.0"
5
install-command: "sudo apt install jq && pip install -r requirements.txt"
6
payload:
7
- "./*"
8
contract:
9
name: "infer"
10
inputs:
11
input_field_1:
12
shape: [-1, 1]
13
type: string
14
profile: text
15
input_field_2:
16
shape: [-1, 1]
17
type: int32
18
profile: numerical
19
outputs:
20
output_field_1:
21
shape: scalar
22
type: int32
23
profile: numerical
24
metadata:
25
experiment: "demo"
26
environment: "kubernetes"
Copied!

kind: Application

The application definition must contain one of the following fields:
  • singular: An object, defining a single-model application;
  • pipeline: A list of objects, defining an application as a pipeline of models.

Singular object

singular object represents an application consisting only of one model. The object must contain the following fields:
  • model: A string, defining a model version. It is expected to be in the form model-name:model-version.
The example below shows how a singular application can be defined.
1
kind: "Application"
2
name: "sample_application"
3
singular:
4
model: "sample_model:1"
Copied!

Pipeline object

pipeline represents a list of stages, representing models.
stage object must contain the following fields:
  • model: A string defining a model version. It is expected to be in the form model-name:model-version.
stage object can contain the following fields:
  • weight: A number defining the weight of the model. All models' weights in a stage must add up to 100.
The example below shows how a pipeline application can be defined.
1
kind: Application
2
name: sample-claims-app
3
pipeline:
4
- - model: "claims-preprocessing:1"
5
- - model: "claims-model:1"
6
weight: 80
7
- model: "claims-model:2"
8
weight: 20
Copied!
In this application, 100% of the traffic will be forwarded to the claims-preprocessing:1 model version and the output will be fed into claims-model. 80% of the traffic will go to the claims-model:1 model version, 20% of the traffic will go to the claims-model:2 model version.

kind: DeploymentConfiguration

The DeploymentConfiguration resource definition can contain the following fields:
  • hpa: An object defining HorizontalPodAutoscalerSpec​
  • container: An object defining settings applied on a container level
  • deployment: An object defining settings applied on a deployment level
  • pod: An object defining settings applied on a pod level

HPA object

The hpa object closely resembles the Kubernetes HorizontalPodAutoscalerSpec object
The hpa object must contain:
  • minReplicas : minReplicas is the lower limit for the number of replicas to which the autoscaler can scale down.
  • maxReplicas : integer, upper limit for the number of pods that can be set by the autoscaler; cannot be smaller than minReplicas.
  • cpuUtilization : integer from 1 to 100, target average CPU utilization (represented as a percentage of requested CPU) over all the pods; if not specified the default autoscaling policy will be used.

Container object

The container object can contain:
  • resources : object with limits and requests fields. Closely resembles the k8s ResourceRequirements object
  • env : object with string keys and string values which is used to set environment variables.

Pod object

The hpa object is similar to the Kubernetes PodSpec object.
The pod object can contain
  • nodeSelector : selector which must be true for the pod to fit on a node. Selector which must match a node's labels for the pod to be scheduled on that node. More info.
  • affinity : pod's scheduling constraints. Represented by an Affinity object.
  • tolerations : array of Tolerations.

Deployment object

The deployment object must contain:
  • replicaCount : integer, number of desired pods. This is a pointer to distinguish between explicit zero and not specified. Defaults to 1.

Example

The example below shows how a deployment configuration can be defined.
1
kind: DeploymentConfiguration
2
name: cool-deployment-config
3
hpa:
4
minReplicas: 2
5
maxReplicas: 10
6
cpuUtilization: 80
7
deployment:
8
replicaCount: 4
9
container:
10
resources:
11
limits:
12
cpu: 500m
13
memory: 4G
14
requests:
15
cpu: 250m
16
memory: 2G
17
env:
18
foo: bar
19
pod:
20
nodeSelector:
21
im: a map
22
foo: bar
23
affinity:
24
nodeAffinity:
25
requiredDuringSchedulingIgnoredDuringExecution:
26
nodeSelectorTerms:
27
- matchExpressions:
28
- key: exp1
29
operator: Exists
30
matchFields:
31
- key: fields1
32
operator: Exists
33
preferredDuringSchedulingIgnoredDuringExecution:
34
- preference:
35
matchExpressions:
36
- key: exp2
37
operator: NotIn
38
values:
39
- aaaa
40
- bvzv
41
- czxc
42
matchFields:
43
- key: fields3
44
operator: NotIn
45
values:
46
- aaa
47
- cccc
48
- zxcc
49
weight: 100
50
podAffinity:
51
requiredDuringSchedulingIgnoredDuringExecution:
52
- labelSelector:
53
matchExpressions:
54
- key: value
55
operator: Exists
56
- key: key
57
operator: NotIn
58
values:
59
- a
60
- b
61
namespaces:
62
- namespace1
63
topologyKey: top
64
preferredDuringSchedulingIgnoredDuringExecution:
65
- weight: 100
66
podAffinityTerm:
67
labelSelector:
68
matchLabels:
69
key: a
70
matchExpressions:
71
- key: key1
72
operator: In
73
values:
74
- a
75
- b
76
- key: value2
77
operator: NotIn
78
values:
79
- b
80
namespaces:
81
- namespace2
82
topologyKey: topo_valur
83
podAntiAffinity:
84
requiredDuringSchedulingIgnoredDuringExecution:
85
- labelSelector:
86
matchExpressions:
87
- key: value
88
operator: Exists
89
- key: key2
90
operator: NotIn
91
values:
92
- a
93
- b
94
- key: key3
95
operator: DoesNotExist
96
namespaces:
97
- namespace1
98
topologyKey: top
99
preferredDuringSchedulingIgnoredDuringExecution:
100
- weight: 100
101
podAffinityTerm:
102
labelSelector:
103
matchLabels:
104
key: a
105
matchExpressions:
106
- key: key
107
operator: In
108
values:
109
- a
110
- b
111
- key: key2
112
operator: NotIn
113
values:
114
- b
115
namespaces:
116
- namespace2
117
topologyKey: toptop
118
tolerations:
119
- effect: PreferNoSchedule
120
key: equalToleration
121
tolerationSeconds: 30
122
operator: Equal
123
value: kek
124
- key: equalToleration
125
operator: Exists
126
effect: PreferNoSchedule
127
tolerationSeconds: 30
Copied!
Last modified 3mo ago