Adding alerts using Service-Level Objectives (SLOs) in Python
Autometrics makes it easy to add Prometheus alerts using Service-Level Objectives (SLOs) to a function or group of functions.
This works using pre-defined Prometheus alerting rules, which can be loaded via the rule_files field in your Prometheus configuration. By default, most of the recording rules are dormant. They are enabled by specific metric labels that can be automatically attached by autometrics.
Pre-requisites
-
Make sure you have set up and initialized Autometrics libraries as described in the quickstart section.
-
Add the pre-configured Prometheus rules file to your Prometheus configuration. This file is located in the Autometrics shared repository (opens in a new tab)
Usage
Create Objectives with your desired SLOs
To use Autometrics SLOs and alerts, create one or multiple Objectives based on the function(s) success rate and/or latency, as shown below.
from autometrics import autometrics
from autometrics.objectives import Objective, ObjectiveLatency, ObjectivePercentile
API_SLO = Objective(
"api_slo",
success_rate=ObjectivePercentile.P99_9,
latency=(ObjectiveLatency.Ms250, ObjectivePercentile.P99),
)Add Objectives to the autometrics decorator
The Objective (API_SLO in this case) can be passed as an argument to the
autometrics decorator to include the given function in that objective.
You can group multiple functions into a single objective by passing the same objective to multiple functions.
from autometrics import autometrics
from autometrics.objectives import Objective, ObjectiveLatency, ObjectivePercentile
API_SLO = Objective(
"api_slo",
success_rate=ObjectivePercentile.P99_9,
latency=(ObjectiveLatency.Ms250, ObjectivePercentile.P99),
)
@autometrics(objective=API_SLO)
def create_user():
# ...
@autometrics(objective=API_SLO)
def get_user():
# ...
@autometrics(objective=API_SLO)
def delete_user():
# ...Viewing SLOs and Alerts
Once you've added objectives to your code, you can use the Autometrics Service-Level Objectives (SLO) Dashboards (opens in a new tab) to visualize the current status of your objective(s).