Skip to content

Expose Application Metrics

On this page you will learn how to publish your own metrics from an Application and have the platform scrape them.

Available Guides

Overview

Why Publish Your Own Metrics

The system metrics of a workload, such as CPU, memory and restarts, tell you that it is alive. They do not tell you that it is doing its job.

Only your Application knows the business-level signals, for example;

  • A sink that is rejecting writes
  • A consumer that is falling behind its queue

If you publish these as metrics, you can see the problem before data goes missing. Without this feature, the only way to do it is a workaround built separately into each Application.

How It Works

  1. Your Application serves its metrics on an HTTP endpoint, using a standard Prometheus library.
  2. You declare the port and path of that endpoint in the app.yaml.
  3. The platform scrapes the endpoint into its metrics backend, together with the system metrics of your workload.

You do not need a sidecar, and you do not need to configure Prometheus.

What You Need to Know

Question Answer
What format must the Application expose? The standard Prometheus exposition format on an HTTP endpoint. Any language with a Prometheus client library works.
Which Application types can use it? Applications and Docker Apps. Any Application with a system section in its app.yaml.
Does it work on edge and air-gapped clusters? Yes. The scraping happens inside the cluster by the local metrics agent, so the metrics surface the same way system telemetry does.
Does it change my existing Applications? No. The setting is optional. Applications that do not declare it behave exactly as before.

Guides

Add Metrics to Your Application

The examples use Python and the standard prometheus_client library. There is nothing Kelvin specific in the code.

  1. Add prometheus_client to the requirements.txt of your Application.

    requirements.txt Example
    1
    prometheus_client

  2. Define your metrics and start the metrics server once, when the Application starts.

    Python Example
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
    from kelvin.application import KelvinApp
from prometheus_client import Counter, Gauge, start_http_server

app = KelvinApp()

# Define the metrics
messages_published = Counter("messages_published_total", "Messages published to the sink")
publish_errors = Counter("publish_errors_total", "Failed publishes")
queue_lag = Gauge("queue_lag_seconds", "How far behind the queue this consumer is")

# Start the metrics server on port 9090
start_http_server(9090)

# Update the metrics in the processing of your Application
@app.timer(interval=10)
async def process():
    messages_published.inc()
    queue_lag.set(compute_lag())

app.run()

    Use a Counter for something that only goes up, such as the number of messages or errors. Use a Gauge for a value that goes up and down, such as a lag or a queue size.

Declare the Metrics in app.yaml

Tell the platform where to scrape. Add metrics to the system section of the app.yaml, using the same port you started the metrics server on.

app.yaml Example
1
2
3
4
5
6
7
8
9
spec_version: 5.0.0

  ...

defaults:
  system:
    metrics:
      port: 9090
      path: /metrics
Field Required Description
port required The port serving the metrics. Range: 1--65535.
path optional The HTTP path of the metrics endpoint. It must start with /. Defaults to /metrics.

Note

port is only required when you add metrics. If you do not add it, the Application behaves as before.

Test the Metrics Locally

Run your Application on your computer and open the metrics address in a browser.

Metrics Address
1
http://localhost:9090/metrics

You will see your metrics in text, for example messages_published_total and queue_lag_seconds. If you see them, the platform can scrape them once the Application is deployed.