Including Data Types
The following commands require an active KSDK session
Check the Quickstart guide on how to login.
Data types are a crucial detail in the development and execution of applications.
They allow users to abstract their business logic into data-centric structures that can be woven into their apps and system.
Kelvin SDK provides, by default, a set of primitive datatypes.
Default Data Types¶
Floats (32, 64 bits)¶
app:
kelvin:
outputs:
- name: my_metric
data_type: raw.float32 # or raw.float64
targets:
- asset_names: ["emulation"]
workload_names: ["my-app"]
...
from kelvin.sdk.datatype import Float32
f = Float32()
f.header.asset_name = "emulation"
f.header.name = "my_metric"
f.value = 100.2
self.emit(f)
# or
message = make_message(
"raw.float32",
"my_metric",
_asset_name="emulation",
value=3.14
)
self.emit(message)
from kelvin.sdk.datatype import Float64
f = Float64()
f.header.asset_name = "emulation"
f.header.name = "my_metric"
f.value = 100.2
self.emit(f)
# or
message = make_message(
"raw.float64",
"my_metric",
_asset_name="emulation",
value=3.14
)
self.emit(message)
Integers (8, 16, 32, 64 bits)¶
app:
kelvin:
outputs:
- name: my_metric
data_type: raw.int8 # or raw.int16, raw.int32, raw.int64
targets:
- asset_names: ["emulation"]
workload_names: ["my-app"]
...
from kelvin.sdk.datatype import Int8
i = Int8()
i.header.asset_name = "emulation"
i.header.name = "my_metric"
i.value = 100
self.emit(i)
# or
message = make_message(
"raw.int8",
"my_metric",
_asset_name="emulation",
value=3
)
self.emit(message)
from kelvin.sdk.datatype import Int16
i = Int16()
i.header.asset_name = "emulation"
i.header.name = "my_metric"
i.value = 100
self.emit(i)
# or
message = make_message(
"raw.int16",
"my_metric",
_asset_name="emulation",
value=3
)
self.emit(message)
from kelvin.sdk.datatype import Int32
i = Int32()
i.header.asset_name = "emulation"
i.header.name = "my_metric"
i.value = 100
self.emit(i)
# or
message = make_message(
"raw.int32",
"my_metric",
_asset_name="emulation",
value=3
)
self.emit(message)
from kelvin.sdk.datatype import Int64
i = Int64()
i.header.asset_name = "emulation"
i.header.name = "my_metric"
i.value = 100
self.emit(i)
# or
message = make_message(
"raw.int64",
"my_metric",
_asset_name="emulation",
value=3
)
self.emit(message)
Text¶
app:
kelvin:
outputs:
- name: my_metric
data_type: raw.text
targets:
- asset_names: ["emulation"]
workload_names: ["my-app"]
...
from kelvin.sdk.datatype import Text
t = Text('my_metric')
t.header.asset_name = "emulation"
t.header.name = "my_metric"
t.value = 'this is a text message'
self.emit(t)
# or
message = make_message(
"raw.text",
"my_metric",
_asset_name="emulation",
value="this is a text message"
)
self.emit(message)
Boolean¶
app:
kelvin:
outputs:
- name: my_metric
data_type: raw.boolean
targets:
- asset_names: ["emulation"]
workload_names: ["my-app"]
...
from kelvin.sdk.datatype import Boolean
b = Text('my_metric')
b.header.asset_name = "emulation"
b.header.name = "my_metric"
b.value = False
self.emit(b)
# or
message = make_message(
"raw.boolean",
"my_metric",
_asset_name="emulation",
value=False
)
self.emit(message)
Object¶
In addition to the primitive types, dictionaries are also supported.
The Object type loosens the type restriction around the value, allowing the user to emit a valid JSON value.
app:
kelvin:
outputs:
- name: my_metric
data_type: object
targets:
- asset_names: ["emulation"]
workload_names: ["my-app"]
...
from kelvin.sdk.datatype import Object
o = Object()
o.value = {"special_list": ["item 1", "item 2", "item 3"]}
o.header.asset_name = "emulation"
o.header.name = "my_metric"
self.emit(o)
Creating custom Data Types¶
It is possible to create custom data types to represent your data objects.
In order for applications to be able to recognize the custom data type,
it is necessary to define them in the applications app.yaml datatypes section.
Raw datatypes are provided by default
The raw data types don't need to be defined in this section, since they are loaded by default in the applications.
A more elaborate page on creating and managing data types can be found here:
Creating your first Data type¶
Using the kelvin datatype create command, let's create a complex data type:
# kelvin datatype create <name> --output-dir=<output-dir>
kelvin datatype create central.config --output-dir=myfolder
Success
[kelvin.sdk][2021-07-01 17:22:03][I] Creating spec for data type "central.config"
[kelvin.sdk][2021-07-01 17:22:03][R] Data type "central.config" spec successfully created in "/absolute/path/myfolder/central_config__0-0-1.yml"
Your data type file will then be available on the datatype directory of your application:
.
├── app.yaml
├── ...
├── datatype
└── central_config__0-0-1.yml
├── my_app
└── ....
└── ...
And will have the following out-of-the-box content:
name: central.config
version: 0.0.1
class_name: CentralConfig
description: "central config"
fields:
- name: myfield
description: A default raw 32 bit floating point value.
type: float32
You can now modify the default fields to accommodate the datatype requirements.
Integrating data types into an application¶
While developing the application, the user can include data types from 2 sources:
- System data types - data types present in the application directory.
- Platform data types - data types that have already been created and updated to the platform. Can be downloaded and reused.
Difference in data type sources
By default, kelvin-sdk will try to load data types from the platform.
If the path keyword is specified on a given data type, kelvin-sdk will attempt to load it from the datatype directory.
app:
kelvin:
data_types:
- name: central.config
version: 0.0.1
# If the 'path' variable is set, it will attempt to load it local
path: datatype/central_config__0-0-1.yml
inputs:
- name: input
data_type: raw.float32
sources:
- workload_names: [ injector ]
asset_names: [ emulation ]
outputs:
- name: central
data_type: central.config
targets:
- workload_names: [ consumer ]
asset_names: [ emulation ]
language:
python:
entry_point: my_app.my_app:App
requirements: requirements.txt
type: python
logging_level: INFO
system_packages: [ ]
type: kelvin
info:
description: my-app
name: my-app
title: my-app
version: 1.0.0
spec_version: 2.0.0