Extend JMX Metrics Collection

Pepperdata collects a rich array of metrics from jobs’ and monitored programs’ JMX services. If you need data beyond what Pepperdata automatically collects and makes available, you can extend its metrics collection to JMX (Java Management Extensions) beans exposed by Java-based daemons (such as Kafka, Zookeeper, and Presto) through HTTP or RMI interfaces.

Prerequisites

General understanding of JMX beans.
Knowledge of which JMX beans provide the metrics that you want to collect.

Procedure

Important: You can begin with any host on which Pepperdata is running, but be sure to repeat all the procedure steps on every host where Pepperdata is running.

If Pepperdata is already collecting extended JMX metrics or performing custom program monitoring, skip to step 2 to add new JMX metrics definitions. Otherwise, configure the rules file as described in this step.

Note: The same rules file is used for extended JMX metrics definitions and program matching rules.

Add the property to specify the filename and location for your rules file, pepperdata.agent.program.monitor.configPath, to the Pepperdata site file, pepperdata-site.xml. Be sure to replace your-file-path-and-name with the name you want for your yaml rules file.

By default, the Pepperdata site file, pepperdata-site.xml, is located in /etc/pepperdata. If you customized the location, the file is specified by the PD_CONF_DIR environment variable. See Change the Location of pepperdata-site.xml for details.
```
<property>
  <name>pepperdata.agent.program.monitor.configPath</name>
  <value>your-file-path-and-name</value>
</property>
```
Malformed XML files can cause operational errors that can be difficult to debug. To prevent such errors, we recommend that you use a linter, such as xmllint, after you edit any .xml configuration file.
Configure your JMX metrics definitions.
1. Open the rules file for editing.
  - If Pepperdata is already performing extended JMX metrics collection or custom program monitoring, open the existing yaml rules file.
  - Otherwise, create a text file for the filename and location that you specified for the pepperdata.agent.program.monitor.configPath property.
2. Add, delete, or revise the extended JMX metrics definitions; see Syntax: JMX Metrics Definitions.
3. Save your changes and close the file.
Verify that your extended JMX metrics definitions are valid; see Verify and Validate JMX Metrics Definitions.

Important: Do not skip this verification step. If there are errors in any definition or the rules file is invalid in any way, none of the definitions and rules are loaded.
Load the extended JMX metrics definitions into the PepAgent by restarting the PepAgent process on the host.

You can use either the service (if provided by your OS) or systemctl command:
- sudo service pepagentd restart
- sudo systemctl restart pepagentd
Tip: Any time you modify the yaml rules file, you must reload the rules file by restarting PepAgent.
Repeat steps 1 to 4 on every host where Pepperdata is running.

Important: Be sure to repeat steps 1 to 4 on every host where Pepperdata is running. If you skip the configuration process on a host, Pepperdata is unable to perform the extended JMX metrics collection.

Next: Syntax: Extended JMX Metrics Collection