General
Quick Start¶
The fundamental class is kelvin.sdk.client.Client, which authenticates users to the API and provides the main point of entry to the various objects and functions in the Kelvin API.
The first time the client is used, url and username must be provided.
These are stored in the client configuration and are not needed for subsequent uses and may be omitted.
>>> from kelvin.sdk.client import Client
>>> client = Client.from_file(
site='yoyodyne',
url="https://yoyodyne.kelvininc.com",
username="fox.mulder"
)
>>> client
Client(url='https://yoyodyne.kelvininc.com')
The client can now be logged-in by supplying the password:
>>> client.login("trustno1")
>>> client.user.get_current_user()
UserWithPermissions(
created='2020-06-30 10:14:24.396000+00:00',
email='fox.mulder@kelvininc.com',
first_name='Fox',
id=UUID('df528dd8-99dc-4c33-9692-f9120db11eea'),
last_name='Mulder',
permissions=[....],
username='fox.mulder'
The password can also be supplied as a keyword-argument at configuration, e.g. Client.from_file(..., password="trustno1").
If your password changes, use Client.login(...) to update it.
The password is securely stored in the system key-chain (secured by your system user-account) and the password can be reused in subsequent sessions:
>>> from kelvin.sdk.client import Client
>>> client = Client.from_file(site='yoyodyne')
>>> client.user.get_current_user()
...
Optionally, the client can be logged-out, which clears the stored password and invalidates the access token:
>>> client.logout()
>>> client.user.get_current_user()
LoginError: Password/secret required to perform login
Working with Data Types¶
Objects can be retrieved and manipulated using methods that map to endpoints on the API.
For example, the list of Nodes can be retrieved using the
kelvin.sdk.client.model.responses.ACP.list_acp method
>>> from kelvin.sdk.client import ACP
>>> acps = client.node.list_acp()
>>> len(acps)
143
>>> acps.name
['acp_1', 'acp_2', ..., 'acp_143']
>>> node = acps[0]
>>> node
ACP(
cluster_name='kelvin',
created='2020-12-18 15:24:17.740484+00:00',
kelvin_info=None,
manifests_scrape_enabled=None,
manifests_scrape_interval=None,
metadata=None,
metrics_enabled=None,
metrics_scrape_interval=None,
name='acp_143',
network_info=None,
node_name='ip-10-10-11-165',
status_scrape_enabled=None,
status_scrape_interval=None,
system_info=None,
title='ACP 143',
updated='2020-12-18 15:26:13.892391+00:00')
Note
For convenience, endpoint methods for data types are mapped on the client via each model name, e.g.
from kelvin.sdk.client import someModelName
someModelName.some_method_name(*args, **kwargs, client=client)
is equivalent to the more convenient:
client.some_model_name.some_method_name(*args, **kwargs)
The "list" endpoints also support filtering server-side, e.g. retrieve all workloads with a particular acp_name:
>>> workloads = client.workload.list_workload(acp_name="my-node")
>>> {*workloads.acp_name}
{'my-node'}
Flexible client-side filtering is also possible via the extended "deep" list class in which results are returned.
Using "array"-style filtering (similar to numpy.ndarray):
>>> workloads = client.workload.list_workload()
>>> workloads_1 = workloads[(workloads.acp_name == "my-node") & workload.enabled]
>>> {*workloads_1.acp_name}
{'my-node'}
>>> {*workloads_1.enabled}
{True}
Using "function"-style filtering, where __ translates to attribute-access:
>>> acp_status = client.acp_status.list_acp_status()
>>> acp_status_not_connected = acp_status(status__state="no_connection")
>>> {*acp_status_not_connected[0].status.state}
{'no_connection'}
Using Time-Series Data¶
Kelvin SDK Client can access time-series data that has been stored in the Cloud Historian.
This data can be obtained using the storage endpoints.
Time-series data can be retrieved for a single metric (node+source+key) over a time-range using
kelvin.sdk.client.model.responses.Storage.get_historian_metric_range or for multiple metrics via
kelvin.sdk.client.model.responses.Storage.get_historian_metric_advanced_range:
>>> from datetime import timedelta
>>> from kelvin.sdk.client import Client
>>> from kelvin.sdk.client.io import storage_to_dataframe
>>> client = Client.from_file(site="my-site")
>>> metrics = client.storage.list_historian_metric(acp_name="my-node", source="my-app")
>>> len(metrics)
5
>>> metrics.key
[
'ns=2;s=avg_temp',
'ns=2;s=avg_pressure',
'ns=2;s=avg_voltage',
'ns=2;s=avg_current',
'ns=2;s=avg_power',
]
>>> metric = metrics[0]
>>> metric.key
'ns=2;s=avg_temp'
>>> # retrieve single metric
>>> now = datetime.datetime.now()
>>> start_time, end_time = now - timedelta(minutes=5), now
>>> data = client.storage.get_historian_metric_range(
... acp_name=metric.acp_name,
... source=metric.source,
... key=metric.key,
... type=metric.type,
... start_time=start_time,
... end_time=now,
... )
>>> next(data) # get one value from stream
StorageData(
acp_name='my-node',
key='ns=2;s=avg_temp',
payload={'value': 100.01},
source='my-app',
timestamp=1574395664434316032,
_insertion_timestamp=1574395664436316320)
>>> df = storage_to_dataframe(data)
>>> df
acp_name source timestamp key
my-node my-app 2020-07-16 09:50:37.867587584+10:00 ns=2;s=avg_temp 95.6
2020-07-16 09:50:44.867924736+10:00 ns=2;s=avg_temp 95.2
2020-07-16 09:50:46.867818496+10:00 ns=2;s=avg_temp 96.0
2020-07-16 09:50:47.867252992+10:00 ns=2;s=avg_temp 96.6
2020-07-16 09:50:49.867187712+10:00 ns=2;s=avg_temp 95.3
2020-07-16 09:50:51.867239168+10:00 ns=2;s=avg_temp 95.8
2020-07-16 09:50:53.867053056+10:00 ns=2;s=avg_temp 95.2
...
>>> # retrieve multiple metrics over the same time window
>>> data = client.storage.get_historian_metric_advanced_range(dict(
... selectors=metrics, start_time=str(start_time), end_time=str(end_time),
... ))
>>> df = storage_to_dataframe(data)
acp_name source timestamp key
my-node my-app 2020-07-16 09:50:37.867587584+10:00 ns=2;s=avg_temp 95.6
2020-07-16 09:50:44.867924736+10:00 ns=2;s=avg_temp 95.2
2020-07-16 09:50:46.867818496+10:00 ns=2;s=avg_temp 96.0
2020-07-16 09:50:47.867252992+10:00 ns=2;s=avg_temp 96.6
2020-07-16 09:50:49.867187712+10:00 ns=2;s=avg_temp 95.3
2020-07-16 09:50:51.867239168+10:00 ns=2;s=avg_temp 95.8
2020-07-16 09:50:53.867053056+10:00 ns=2;s=avg_temp 95.2
...
2020-07-16 09:50:37.867587584+10:00 ns=2;s=avg_power 19.3
2020-07-16 09:50:44.867924736+10:00 ns=2;s=avg_power 18.2
2020-07-16 09:50:46.867818496+10:00 ns=2;s=avg_power 19.8
2020-07-16 09:50:47.867252992+10:00 ns=2;s=avg_power 19.9
2020-07-16 09:50:49.867187712+10:00 ns=2;s=avg_power 18.9
2020-07-16 09:50:51.867239168+10:00 ns=2;s=avg_power 19.8
2020-07-16 09:50:53.867053056+10:00 ns=2;s=avg_power 19.5
...
The data-frame can now be exported via any of the pandas.DataFrame.to_*
functions, e.g.
to_csv
or
to_json.
Note
Data returned by the API is "streamed" in chunks from the server, and
data in the example above is an iterator, not a list. If the raw storage
rows are required without conversion to a dataframe, use rows = [*data]
to collect all entries from the stream.
In addition to start_time and end_time, arguments can be provided to modify
the returned data, e.g.
-
aggallows you to choose an aggregation:- The available aggregations for numeric types
raw.int32,raw.int64,raw.float32andraw.float64are:none,count,distinct,integral,mean,median,mode,spread,stddev,sum - The available aggregations for non-numeric types (
raw.textor any other) are:count,distinctandmode.
- The available aggregations for numeric types
-
time_bucket: The window of data to aggregate, e.g.5m,1h,1w(see https://golang.org/pkg/time/#ParseDuration for the general format) time_shift: The offset for each window. Set to be the same astime_bucketfor a tumbling window, or less for overlapping/rolling window.fillallows you to fill missing points from a time bucket. It might be one of:none(default);null;linear(performs a linear regression);previous(uses the previous non-empty value); or an int.
Configuration¶
The Kelvin SDK Client can be configured to persist user information and other site-specific configuration in a YAML file.
The default location for this file is ${XDG_CONFIG_HOME}/kelvin/client.yaml (default ${XDG_CONFIG_HOME} is ${HOME}/.config).
The default configuration file is:
client:
url: # Base URL of API
realm_name: kelvin # KeyCloak realm
username: # User name
token: # Access token
retries: 3 # Number of retries
timeout: # Request timeout: (connect, read)
- 6.05
- 60.0
gzip: false # Use gzip on requests
verify: true # Verify SSL/TLS certificates
history: 5 # Number of requests to retain in history
sites: # Site-specific overrides
Note
The sites section can contain site-specific overrides (e.g. for
username or url) which provide configuration sets that can be used
to create kelvin.sdk.client.Client instances with specific setup.
The configuration file can be updated manually in a text-editor, however it is recommended
that the configuration be managed via the config object and update the file with
kelvin.sdk.client.ClientConfiguration.to_file:
>>> client = Client.from_file(site="yoyodyne")
>>> client.config.username
'fox.mulder'
>>> client.config.username = "dana.scully"
>>> client.config.to_file()
>>> client = Client.from_file(site="yoyodyne")
>>> client.config.username
'dana.scully'
Troubleshooting¶
If you run into issues, there are some methods for determining what might be causing the problem.
Request History¶
To debug a client request, it is possible to use the request history to understand better the request that was made, and the response from the server.
For example, to get the last request made:
>>> acps = client.node.list_acp()
>>> client.history[-1]
History(
url='https://demo.kelvininc.com/api/v4/acps/list?page_size=20&starting_after=qamm-testing-demo',
method='GET',
body=None,
start=datetime.datetime(2021, 1, 19, 19, 4, 59, 806661, tzinfo=datetime.timezone.utc),
end=datetime.datetime(2021, 1, 19, 19, 4, 59, 928072, tzinfo=datetime.timezone.utc),
request=<PreparedRequest [GET]>,
response=<Response [200]>
)
>>> *_, response = client.history[-1]
>>> response.json()
{'data': [{'name': 'my-node',
'title': 'my-node',
'metadata': None,
'created': '2020-11-26T15:49:01.482281Z',
'updated': '2020-11-26T15:50:16.65293Z'},
...]}
The posted data is also accessible:
>>> node = node[-1]
>>> node.title = "My ACP"
>>> node.update_acp()
>>> client.history[-1]
History(
url='https://demo.kelvininc.com/api/v4/acps/x-wing/update',
method='POST',
body='...',
start=datetime.datetime(2021, 1, 19, 19, 9, 4, 256073, tzinfo=datetime.timezone.utc),
end=datetime.datetime(2021, 1, 19, 19, 9, 4, 435780, tzinfo=datetime.timezone.utc),
request=<PreparedRequest [POST]>,
response=<Response [200]>
)
Each history entry is a tuple containing:
- Request URL
- HTTP method (
GET,POST, etc) - Request body (typically a JSON message or empty for a
GET) - Request start time
- Request end time
- Response object (
requests.Response)
The number of history entries can be configured by the history
configuration item.