/m (Metrics)
The Pepperdata REST API resource for metrics, series, and filters is /your-realm-name/api/m
.
On This Page
Metrics API Query String Arguments
querystring key | description |
---|---|
s | Start time, in the format YYYY/MM/DD-HH:mm . |
e | End time, in the format YYYY/MM/DD-HH:mm . |
tzo | Time zone, in offset from UTC hours. |
m | Metric to access. For the list of metrics, refer to the Metric Definitions page on your dashboard (on the navigation bar, click Help > Metric Definitions); click or hover over the metric you want to see the metric name, which appears after the #m= part of the URL. |
sample | (required if targetpoints not specified; n/a otherwise) Sample interval (time between points returned), in milliseconds. WARNING: A small value over a long time period is expensive. The minimum value is 5000 (5 seconds). For a query that lasts a few hours, we recommend a value of 10000 (10 seconds). |
targetpoints | (required if sample not specified; n/a otherwise) Target number of points to return. The actual number of points returned can be greater or fewer depending on the sample size that is cached. |
ms | (optional, default=20) Maximum Series to return. If the number of series exceeds this value, we show the top n series and include the remaining series as aggregates: sum_of_the_others, max_of_the_others, average_of_all_matched_series. |
nohostsreporting | (optional, default=off) Include or omit the hostsreporting in the response. To omit hostsreporting, set to 1 . |
omitpoints | (optional, default=off) Include or omit datapoints in the response. To omit all datapoints in the response, set to all . |
downsampler | (optional, default=off) Flag to split a single series into percentiles by time. To split a series, set to percentile . NOTE: The percentile downsampler applies only to queries that would otherwise return a single series. |
format | (optional, default=json ) Format. For the fastest API performance, use the JSON format (lowercase json ). To get the query in CSV format, set to (lowercase) csv . |
omitaggregates | (optional, default=off) Include or omit aggregate series—sum_of_the_others, max_of_the_others, and average_of_all_matched_series—in the response. To omit aggregate series, set to 1 . |
dataonly | (optional, default=off) Include or omit progress updates and other performance instrumentation data in the response. To omit progress updates and instrumentation data, set to 1 . |
Metrics Breakdowns and Filters
You can add series breakdowns and/or filters to the metrics’ requests.
To get the breakdown of all the results, use *
for the querystring value.
To specify a filter, use the specific string.
For example, to breakdown results by hosts, add the querystring &h=*
.
To filter by a given hostname, for example, “my-host”, add the querystring &h=my-host
.
To perform a regex match, add the querystring &h=/.*/
.
You can specify multiple breakdowns.
For example, for all jobs on a specific host, my-host, add the querystring, &h=my-host&j=*
.
The table describes the breakdowns and filters options.
querystring key | description |
---|---|
h | host |
u | user |
q | queue |
c | control domain |
j | job sequence id |
tt | task type |
t | task The only valid value is * . This querystring key cannot be used by itself. It must be used with another filter. For example, to get all the tasks for a job, use &j={JOBID}&t=* . |
d | device |
Data Format of the Metrics API
The Pepperdata REST API metrics resource returns a single JSON object, defined as follows:
- expectedCount: Expected number of progress chunks.
- (omitted if
dataonly=1
) progress: Array of progress updates during the query runtime. - (omitted if
dataonly=1
) instrumentation: Debug information about the query, such as timing numbers. - data: Data payload.
- allseries: Array of timeseries returned timeseries.
- {timeseries}: timeseries object inside allseries.
- dataPoints: Array of arrays of the form
[[timestamp, value], [timestamp,value] ...]]
- summaryStats: Object that contains the summaries for each series, such as area, average (mean), min, p05, max, and p95.
- name: Object that contains name details about the series; varies by breakdown/filter.
- seriesId: Unique name for this series.
- dataPoints: Array of arrays of the form
- {timeseries}: timeseries object inside allseries.
- sampleMillis: Distance between points, in milliseconds.
- startTimeMillis: Start timestamp.
- hostsReporting: Object with the number of hosts that have finished reporting; helps in determining whether data collection is complete.
- expectedHosts: Number of hosts expected.
- hostCounts: Array of arrays that contain the timestamp and number of hosts reported at that time.
- allseries: Array of timeseries returned timeseries.
Example (Python): Metrics API
The commented Python example shows how to make an HTTP request to the Pepperdata REST API resource for retrieving the Pepperdata dashboard metrics data.
# Pepperdata REST API Python example for retrieving metrics.
# Sections are denoted by '#' characters.
########################### IMPORTS ###########################
# Not required, but recommended for date and time manipulation.
# Required in this example.
import datetime
# To get this use the following: `pip install requests`
# If you do not have the pip Python package management system,
# install it from the python.org website:
# https://packaging.python.org/tutorials/installing-packages/
import requests
import os
# The authorization process (and therefore the required code imports) is
# different for recently-generated keys and legacy keys
# (keys generated before late August, 2017). Recently-generated
# keys are listed in the Active Keys section of the API Keys page
# (https://dashboard.pepperdata.com/account/apikeys); legacy keys are
# listed in the Legacy Keys section.
#
# **** IMPORTANT ****
#
# Be sure to use the correct commands (comment/uncomment the applicable
# lines) wherever you see "Active Key" or "Legacy Key" in the comments in
# this script.
#
# Legacy keys are DEPRECATED as of 2022 Oct 19. If you have one, generate a new
# key at https://dashboard.pepperdata.com/account/apikeys and follow the
# Active Key instructions.
#
# *******************
# (Legacy Key) Required
#from requests.auth import HTTPBasicAuth
####################### DEFINE GLOBALS ########################
# Name of the your cluster/realm configured on Pepperdata
# **** IMPORTANT ****
#
# Before you run this Python example script, be sure to assign the correct
# realm name to the REALM_NAME environment variable by exporting an
# appropriate value for 'your-realm-name':
#
# export REALM_NAME='your-realm-name'
#
# *******************
REALM_NAME = os.environ['REALM_NAME']
# Base url where the Pepperdata dashboard REST API metrics
# resource for your realm is hosted
base_url = 'https://dashboard.pepperdata.com/{0}/api/m'.format(REALM_NAME)
# For Pepperdata-hosted dashboards (the default implementation),
# API access requires authentication via an API key.
# To obtain an API key, go to the following web page:
# https://dashboard.pepperdata.com/account/apikeys
#
# The "ID" is the portion of the API key before the ":"
# The password (secret) token is the portion of the API key after the ":"
# **** IMPORTANT ****
#
# Before you run this Python example script, be sure to assign the correct
# values to the API_KEY_ID and API_KEY_SECRET environment variables by
# exporting the correct values for 'your-api-key-id' and 'your-api-key-secret':
#
# export API_KEY_ID='your-api-key-id'
# export API_KEY_SECRET='your-api-key-secret'
#
# *******************
API_KEY_ID = os.environ['API_KEY_ID']
API_KEY_SECRET = os.environ['API_KEY_SECRET']
# The metrics API uses this format for time OR default Unix time.
pepperdata_time_format = "%Y/%m/%d-%H:%M"
# Although there is no maximum for the time range that you can specify,
# we recommend setting it to a maximum of 14 days.
# This example uses 1 day for the time range.
end_time = datetime.datetime.now()
start_time = end_time - datetime.timedelta(days=1)
# The distance between two data points in the returned time series in number of milliseconds.
sample_millis = 60000
# To determine the name of a metric on a chart, hover your mouse over the JSON
# export option link. Use the link's m={metric_url_name} parameter.
# The full list of metrics is on the Metrics Definitions page on the dashboard,
# http://dashboard.pepperdata.com/your-realm-name/metricdefinitions.
# This example use the "Active Tasks" metric, with m='tasks'.
metric_url_name = 'tasks'
# Time Zone, offset from UTC
time_zone_hour_offset = -7
# Output format: JSON or CSV. For fastest API performance, use JSON.
output_format = 'JSON'
# Maximum number of series (results) to return for the query
max_series = 20
# Assemble all the parameters to easily pass to the HTTP request portion.
# This example changes from the datetime format to the Pepperdata format
# by using strftime(). You could use Unix time instead.
pepperdata_params = {
's': start_time.strftime(pepperdata_time_format),
'e': end_time.strftime(pepperdata_time_format),
'sample': sample_millis,
'm': metric_url_name,
'tzo': time_zone_hour_offset,
'format': output_format,
'ms': max_series
}
# To list the series by host/user/job/etc., add breakdowns to the parameters.
# This example breaks down the results by user ('u') and job ('j').
# For the complete list of parameters, refer to the following help page
# on your dashboard:
# https://dashboard.pepperdata.com/your-realm-name/metricdefinitions?print=1
pepperdata_params['u'] = '*'
pepperdata_params['j'] = '*'
# (Active Key) Required header
headers = {
'Accept': 'application/json',
'Authorization': 'PDAPI ' + API_KEY_ID + ":" + API_KEY_SECRET
}
# (Legacy Key) Required header
#headers = {'Accept': 'application/json'}
########### MAKE THE REQUEST AND PARSE THE RESPONSE ###########
# Send the request to the Pepperdata REST API metrics resource,
# passing the base url, all the parameters, the username and password,
# and the required header, which you defined previously.
# (Active Key) request:
try:
response = requests.get(base_url, pepperdata_params,
headers=headers)
# (Legacy Key) request:
try:
# response = requests.get(base_url, pepperdata_params, auth=HTTPBasicAuth(API_KEY_ID, API_KEY_SECRET),
# headers=headers)
except:
print ("Error receiving response from server.")
# Parse the response as JSON, as specified by the output_format, above.
json_response = response.json()
# Store the data portion of the response. Everything else in the response
# is the progress updates as the database loaded the response.
json_data = json_response['data']
# Parse the data portion of the response, and store all the returned series
# in the all_series object.
all_series = json_data["allSeries"]
# Get the first point in the series, from the all_series object that
# stores the point elements as arrays of format [timestamp, data]. In this case,
# the point is [timestamp, active_tasks_at_timestamp]. The entirety of
# dataPoints is a list of these two element lists.
first_series_points = all_series[0]["dataPoints"]
# Get the object that contains the summary statistics of the series, including
# the following statistics:
# Area, Average, Minimum, Percentile 05, Maximum, and Percentile 95 over
# the entire series.
first_series_summary_stats = all_series[0]["summaryStats"]
# Get the unique series ID for this particular query.
first_series_id = all_series[0]["seriesId"]
# Get the map of name components for this series. If there are filters on
# this query, it will let you know that this series is a breakdown by a
# particular host and job, whose names are included here.
first_series_name_components = all_series[0]["name"]["seriesNameMap"]
# Print the output nicely.
for series in all_series:
print "{0} {1}".format(series["summaryStats"]["avg"], series["seriesId"])