File Storage

Overview

The File Storage feature is for storing any type of files on the Kelvin Platform for access through the Kelvin API endpoint.

All files are accessible by any authorized user to the Platform with the correct Roles & Permissions.

The files can not be read remotely. A local copy must be downloaded to read the file.

Some common uses for the file system are for;

• Machine Learning Kelvin SmartApps™ and Docker Apps can load the latest model weights files using filestorage and in the case of Kelvin SmartApps™, you can set the filename in the configuration parameters.
• Kelvin SmartApps™, Docker Apps and Workloads have a sharable globally accessible file system
• Decoupling of data files so that Kelvin SmartApps™, Docker Apps and Workloads can be stateless

List Files in Storage

In this example we will get the list of all CSV files stored on the Kelvin Platform.

There are also options to filter the response for on source and on the exact filename.

Kelvin APIKelvin API Client (Python)

curl -X 'POST' \
'https://<url.kelvin.ai>/api/v4/filestorage/list?pagination_type=cursor&page_size=20&direction=asc' \
-H 'Authorization: Bearer <Your Current Token>' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"search": [
    "csv"
]
}'

The response will look like this;

{
  "pagination": {
    "next_page": "W3siS2V5IjoiaWQiLCJWYWx1ZSI6IjlmZmQxNjU3LThhMjEtNDQzNS04MTY1LTQwYjcwNmU2ZTlhNyJ9XQ==",
    "previous_page": null
  },
  "data": [
    {
      "file_id": "0107549c-2069-4824-bc4f-30c7bc94172e",
      "file_name": "publisher-example (7).csv",
      "file_size": 603,
      "checksum": "7de7ebb64b70a2b89c6b25c9eb0e23372c8f902390a8448cc1aeb3da70e4a7e8",
      "created": "2024-03-06T16:20:59.416391Z",
      "source": "krn:user:ricardo.goncalves@kelvininc.com",
      "metadata": {}
    },
    {
      "file_id": "0adecb9e-1a50-4459-946d-cb73fc7efe98",
      "file_name": "publisher-example.csv",
      "file_size": 603,
      "checksum": "7de7ebb64b70a2b89c6b25c9eb0e23372c8f902390a8448cc1aeb3da70e4a7e8",
      "created": "2024-03-12T10:31:55.190712Z",
      "source": "krn:user:daniel.carvalho@kelvininc.com",
      "metadata": {
        "allowPlayback": true
      }
    }
  ]
}

from kelvin.api.client import Client

# Login
client = Client(config={"url": "https://<url.kelvin.ai>", "username": "<your_username>"})
client.login(password="<your_password>")

# List Files in Forestorage
response = client.filestorage.list_files(data={
    'search': ['csv']
    })

print(response)

You will see the output like this;

data=[FileStorage(file_id='0107549c-2069-4824-bc4f-30c7bc94172e', file_name='publisher-example.csv', file_size=603, checksum='7de7ebb64b70a2b89c6b25c9eb0e23372c8f902390a8448cc1aeb3da70e4a7e8', source='krn:user:demo@kelvin.ai', created=datetime.datetime(2024, 3, 6, 16, 20, 59, 416391, tzinfo=datetime.timezone.utc), metadata={}), FileStorage(file_id='0adecb9e-1a50-4459-946d-cb73fc7efe98', file_name='streaming_data.csv', file_size=603, checksum='7de7ebb64b70a2b89c6b25c9eb0e23372c8f902390a8448cc1aeb3da70e4a7e8', source='krn:user:demo@kelvin.ai', created=datetime.datetime(2024, 3, 12, 10, 31, 55, 190712, tzinfo=datetime.timezone.utc), metadata={'allowPlayback': True})] pagination=PaginationCursor(next_page=None, previous_page=None)

Upload Files to Storage

In this example we will upload a CSV file to the stored on the Kelvin Platform and receive the confirmation.

There are also options to add custom metadata to help classify and filter file listings.

Kelvin APIKelvin API Client (Python)

curl -X 'POST' \
'https://<url.kelvin.ai>/api/v4/filestorage/upload' \
-H 'Authorization: Bearer <Your Current Token>' \
-H 'accept: application/json' \
-H 'Content-Type: multipart/form-data' \
-F 'file=@/home/ubuntu/test_data.csv;type=text/plain' \
-F 'metadata={"customkey": "Custom metadata information"}'

The response will look like this;

{
"file_id": "66fce65f-d43f-4424-89b8-e1dd01f9193b",
"file_name": "test_data.csv",
"file_size": 6,
"checksum": "66a045b452102c59d840ec097d59d9467e13a3f34f6494e539ffd32c1bb35f18",
"created": "2024-03-19T10:29:55.673471Z",
"source": "krn:user:demo@kelvin.ai",
"metadata": {
    "customkey": "Custom metadata information"
}
}

from kelvin.api.client import Client

# Login
client = Client(config={"url": "https://<url.kelvin.ai>", "username": "<your_username>"})
client.login(password="<your_password>")

# Upload file to filestore
response = client.filestorage.upload_file(
    file='/home/ubuntu/model_weights.h5', 
    metadata={"customkey": "Custom metadata information"}
)

print(response)

You will see the output like this;

file_id='d9a5a89b-7a72-4ff3-a7b6-eaefec4875ad' file_name='model_weights.h5' file_size=3066 checksum='1b342f992ac36da9018e84ba81623f3a862d351497bae2a129fe70df02e085ad' source='krn:user:demo@kelvin.ai' created=datetime.datetime(2024, 3, 24, 1, 51, 54, 699950, tzinfo=datetime.timezone.utc) metadata={"customkey": "Custom metadata information"}

Delete Files on Storage

In this example we will delete a file stored on the Kelvin Platform and receive the confirmation.

You can only delete one file at a time. You need to use the id of the file which will be a UUID number. You can not use the filename.

The delete is permanent and not recoverable.

Kelvin APIKelvin API Client (Python)

curl -X 'POST' \
'https://<url.kelvin.ai>/api/v4/filestorage/66fce65f-d43f-4424-89b8-e1dd01f9193b/delete' \
-H 'Authorization: Bearer <Your Current Token>' \
-H 'accept: application/json' \
-d ''

The response code will be 200 or an error code from 4XX.

from kelvin.api.client import Client

# Login
client = Client(config={"url": "https://<url.kelvin.ai>", "username": "<your_username>"})
client.login(password="<your_password>")

# Delete File in Filestorage
response = client.filestorage.delete_file(
    file_id='0107549c-2069-4824-bc4f-30c7bc94172e'
)

print(response)

You will see the output like this;

None

Download Files from Storage

In this example we will download a file from the storage on the Kelvin Platform.

There are also options to add custom metadata to help classify and filter file listings.

Kelvin APIKelvin API Client (Python)

curl -X 'GET' \
'https://<url.kelvin.ai>/api/v4/filestorage/ef636192-e7eb-4f15-82df-2880df880f65/download' \
-H 'Authorization: Bearer <Your Current Token>' \
-H 'accept: application/octet-stream'

The response code will be 200 or an error code from 4XX and the file itself.

This will save the file locally.

from kelvin.api.client import Client

# Login
client = Client(config={"url": "https://<url.kelvin.ai>", "username": "<your_username>"})
client.login(password="<your_password>")

# Download File from Filestorage
 with open("output.csv", "w") as file:
        writer = csv.writer(file)
        for byte in client.filestorage.download_file(file_id="0adecb9e-1a50-4459-946d-cb73fc7efe98"):
            decoded_str = byte.decode("utf-8")
            rows = decoded_str.split("\n")
            for row in rows:
                writer.writerow(row.split(","))

Get Info about Files on Storage

In this example we will retrieve the information about a file in storage on the Kelvin Platform.

You can only delete one file at a time. You need to use the id of the file which will be a UUID number. You can not use the filename.

Kelvin APIKelvin API Client (Python)

curl -X 'GET' \
'https://<url.kelvin.ai>/api/v4/filestorage/ef636192-e7eb-4f15-82df-2880df880f65/get' \
-H 'Authorization: Bearer <Your Current Token>' \
-H 'accept: application/json'

The response will look like this;

{
"file_id": "ef636192-e7eb-4f15-82df-2880df880f65",
"file_name": "test_data.csv",
"file_size": 40,
"checksum": "c9e7404f3c7f77dbb85e273deb48a4e95ed1fd373672fc3c2a6ef44bef2349b7",
"created": "2024-03-19T10:29:55.673471Z",
"source": "krn:user:demo@kelvin.ai",
"metadata": {
    "customkey": "Custom metadata information"
}
}

from kelvin.api.client import Client

# Login
client = Client(config={"url": "https://<url.kelvin.ai>", "username": "<your_username>"})
client.login(password="<your_password>")

# Get Info on File in Filestorage
response = client.filestorage.get_file(
    file_id='0107549c-2069-4824-bc4f-30c7bc94172e'
)

print(response)

You will see the output like this;

file_id='fddbcc17-1393-4721-86ad-21ae1ec2f326' file_name='data.csv' file_size=1375 checksum='462a4133b03109ec4b20eaa002bea303dc7aef33045e963c3dc1860af8eb00bb' source='krn:user:ivo.fernandes@kelvininc.com' created=datetime.datetime(2024, 3, 22, 10, 13, 22, 592889, tzinfo=datetime.timezone.utc) metadata={'allowPlayback': True}