Kelvin SDK Client =================== The Kelvin SDK client provides a Python 3 object wrapper for the Kelvin REST API, mapping the JSON data structures provided by the Kelvin API into Python objects. This tool is licensed under the Kelvin SDK and API Agreement to provide developers and data-scientists access to the Kelvin API and platform. .. toctree:: :maxdepth: 4 :caption: Contents client modules/requests modules/responses Installation ------------ The Kelvin SDK client wheel file can be installed for Python 3.7 and above using ``pip``: .. code-block:: bash $ pip install kelvin-sdk-client Quick Start ----------- The fundamental class is :obj:`kelvin.sdk.client.Client`, which authenticates users to the API and provides the main point of entry to the various objects and functions in the Kelvin API. The first time the client is used, ``url`` and ``username`` must be provided. These are stored in the client configuration and are not needed for subsequent uses and may be omitted. .. code-block:: ipython3 :caption: Initialising the client: from kelvin.sdk.client import Client client = Client.from_file( site='yoyodyne', url="https://yoyodyne.kelvininc.com", username="fox.mulder" ) >>> client Client(url='https://yoyodyne.kelvininc.com') The client can now be logged-in by supplying the password: .. code-block:: ipython3 :caption: Logging-in the client: client.login("trustno1") >>> client.user.me() User( created='2019-10-29 20:50:34.801000+00:00', email='fox.mulder@kelvininc.com', first_name='Fox', id=UUID('f35f15c9-ae91-4165-9610-dd02f4dadb6e'), last_name='Mulder', username='fox.mulder') .. note:: The password can also be supplied as a keyword-argument at configuration, e.g. ``Client.from_file(..., password="trustno1")``. If your password changes, use :meth:`~kelvin.sdk.client.Client.login` to update it. The password is securely stored in the system key-chain (secured by your system user-account) and the password can be reused in subsequent sessions: .. code-block:: ipython3 :caption: Re-connecting to the client: from kelvin.sdk.client import Client client = Client.from_file(site='yoyodyne') >>> client.user.me() ... Optionally, the client can be logged-out, which clears the stored password and invalidates the access token: .. code-block:: ipython3 :caption: Logging-out the client: >>> client.logout() >>> client.user.me() LoginError: Password required to perform login Working with Data Models ------------------------ Objects can be retrieved and manipulated using methods that map to endpoints on the API. For example, the list of ACPs can be retrieved using the :meth:`~kelvin.sdk.client.model.responses.ACP.list_acps` method .. code-block:: ipython3 :caption: Using an endpoint with the client: >>> from kelvin.sdk.client import ACP >>> acps = ACP.list_acps(client=client) # or simply client.acp.list_acps() >>> len(acps) 143 >>> acps.name ['acp_1', 'acp_2', ..., 'acp_143'] >>> acp = acps[0] >>> acp ACP( cluster_name='kelvin', created='2020-03-24 15:47:35.646950+00:00', metadata=None, name='acp_143', node_name='ip-10-10-11-165', title='ACP 143', updated='2020-03-24 15:47:35.646950+00:00') .. note:: For convenience, endpoint methods for data models are mapped on the client via each model name, e.g. .. code-block:: python from kelvin.sdk.client import SomeModelName SomeModelName.some_method_name(*args, **kwargs, client=client) is equivalent to the more convenient: .. code-block:: python client.some_model_name.some_method_name(*args, **kwargs) The "list" endpoints also support filtering server-side, e.g. retrieve all workloads with a particular ``acp_name``: .. code-block:: ipython3 :caption: Filtering results (server-side): >>> workloads = client.workload.list_workloads(acp_name="my-acp") >>> {*workloads.acp_name} {'my-acp'} Flexible client-side filtering is also possible via the extended "deep" list class in which results are returned. Using "array"-style filtering (similar to `numpy.ndarray`): .. code-block:: ipython3 :caption: Filtering results (client-side) - "array-style" filtering: >>> workloads = client.workload.list_workloads() >>> workloads_1 = workloads[(workloads.acp_name == "my-acp") & workload.enabled] >>> {*workloads_1.acp_name} {'my-acp'} >>> {*workloads_1.enabled} {True} Using "function"-style filtering, where ``__`` translates to attribute-access: .. code-block:: ipython3 :caption: Filtering results (client-side) - "function-style" filtering: >>> cluster_nodes = client.k8s_node.list_k8s_cluster_nodes("kelvin") >>> cluster_nodes_1 = cluster_nodes(status__ready="True") >>> {*cluster_nodes_1.status.ready} {'True'} Using Time-Series Data ---------------------- Kelvin Client can access time-series data that has been stored in the Cloud Historian. This data can be obtained using the ``storage`` endpoints. Time-series data can be retrieved for a single acp/key over a time-range using :meth:`~kelvin.sdk.client.model.responses.Storage.get_range_storage`: .. code-block:: ipython3 :caption: Time-series storage-retrieval (single acp/key) >>> from kelvin.sdk.client.io import storage_to_dataframe >>> now = datetime.now() >>> data = client.storage.get_range_storage( ... acp_name="acp_1", ... source="my-app", ... key="raw.float32.gas_flow", ... start_time=now - timedelta(minutes=5), ... end_time=now, ... order="ASC" ... ) >>> next(data) Storage( acp_name='acp_1', key='raw.float32.gas_flow', metadata={}, payload={'value': 100.01}, source='my-app', timestamp=1574395664434316032) >>> df = storage_to_dataframe(data) >>> df.head() metadata source value acp_name key timestamp acp_1 raw.int32.qamm 2019-11-22 04:09:26.235456+00:00 {} my-app 100.01 2019-11-22 04:50:39.796025088+00:00 {} my-app 103.44 2019-11-22 12:28:08.573904896+00:00 {} my-app 132.22 2019-11-22 12:29:07.268737024+00:00 {} my-app 103.32 2019-11-22 19:42:21.693916928+00:00 {} my-app 110.21 .. note:: Data returned by the :meth:`~kelvin.sdk.client.model.responses.Storage.get_range_storage` method is "streamed" in chunks from the server, and ``data`` in the example above is an iterator, not a list. .. _configuration: Configuration ------------- The Kelvin API client can be configured to persist user information and other site-specific configuration in a YAML file. The default location for this file is ``${XDG_CONFIG_HOME}/kelvin/client.yaml`` (default ``${XDG_CONFIG_HOME}`` is ``${HOME}/.config``). The default configuration file is: .. code-block:: yaml client: url: # Base URL of API realm_name: kelvin # KeyCloak realm username: # User name token: # Access token retries: 3 # Number of retries timeout: # Request timeout: (connect, read) - 6.05 - 60.0 gzip: false # Use gzip on requests verify: true # Verify SSL/TLS certificates history: 5 # Number of requests to retain in history sites: # Site-specific overrides .. note:: The ``sites`` section can contain site-specific overrides (e.g. for ``username`` or ``url``) which provide configuration sets that can be used to create :obj:`kelvin.sdk.client.Client` instances with specific setup. The configuration file can be updated manually in a text-editor, however it is recommended that the configuration be managed via the ``config`` object and update the file with :meth:`~kelvin.sdk.client.ClientConfiguration.to_file`: .. code-block:: ipython3 :caption: Updating configuration: >>> client = Client.from_file(site="yoyodyne") >>> client.config.username 'fox.mulder' >>> client.config.username = "dana.scully" >>> client.config.to_file() >>> client = Client.from_file(site="yoyodyne") >>> client.config.username 'dana.scully' Troubleshooting --------------- If you run into issues, there are some methods for determining what might be causing the problem. Request History ~~~~~~~~~~~~~~~ To debug a client request, it is possible to use the request history to understand better the request that was made, and the response from the server. For example, to get the last request made: .. code-block:: ipython3 :caption: Request history (GET) >>> acps = client.acp.list_acps() >>> client.history[-1] ('https://titan.kelvininc.com/api/v4/acps/list?page_size=20&starting_after=qamm-acp-15', 'GET', None, datetime.datetime(2020, 3, 24, 21, 34, 25, 766657, tzinfo=datetime.timezone.utc), datetime.datetime(2020, 3, 24, 21, 34, 26, 31413, tzinfo=datetime.timezone.utc), ) >>> *_, response = client.history[-1] >>> response.json() {'data': [{'name': 'acp_1', 'title': 'ACP 1', 'node_name': 'edge-virtual-machine', 'cluster_name': 'kelvin', 'metadata': None, 'created': '2020-03-21T23:58:14.356754Z', 'updated': '2020-03-21T23:58:14.356754Z'}, ...]} The posted data is also accessible: .. code-block:: ipython3 :caption: Request history (POST) >>> acp = acp[-1] >>> acp.title = "My ACP" >>> acp.update_acp() >>> client.history[-1] ('https://titan.kelvininc.com/api/v3/acps/acp_X/update', 'POST', '{..., "title": "My ACP"}', datetime.datetime(2019, 11, 27, 1, 14, 1, 644912, tzinfo=datetime.timezone.utc), datetime.datetime(2019, 11, 27, 1, 14, 1, 917335, tzinfo=datetime.timezone.utc), ) Each history entry is a tuple containing: - Request URL - HTTP method (``GET``, ``POST``, etc) - Request body (typically a JSON message or empty for a ``GET``) - Request start time - Request end time - Response object (``requests.Response``) The number of history entries can be configured by the ``history`` configuration item. Dry-Run ~~~~~~~ Each request also has a "dry-run" optional argument which will simply show what the request would be sent in the request to the server: .. code-block:: ipython3 :caption: Request history (POST) >>> acp.delete_acp(dry_run=True) {'path': '/api/v3/acps/acp_1/delete', 'method': 'post', 'data': None, 'params': {}, 'files': {}} Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` .. toctree:: :hidden: LICENSE