Prometheus Exporter¶

TNSR includes a Prometheus exporter which supports statistical data from the dataplane (VPP) only. This data is typically collected by a Prometheus server and then visualized with analytics platforms such as Grafana. When active, the service listens for connections on TCP port 9482.

Configuring Prometheus¶

prometheus <namespace> enable:

Enables the Prometheus Exporter service in either the host or dataplane namespace.

Warning

Using the host namespace is more secure as limits the exposure of the service to host OS management networks.

After enabling the service, start it as described in Service Control.

prometheus <namespace> disable:

Disables the Prometheus Exporter service in the given namespace.

prometheus <namespace> filter <regex> [<regex> […]]:

Adds one or more regular expression filters which limit the data exposed by the service. For details about how these filters work, see Prometheus Filtering.

no prometheus <namespace> filter <regex>:

Removes a filter.

Prometheus Filtering¶

The data exposed by Prometheus to clients can be limited by filtering. Limiting in this manner can increase performance by reducing the amount of data sent to clients and the amount that clients must process as a consequence. By default, all data in the dataplane statistics segment is exported which can be sizable on large deployments.

Prometheus filters consist of one or more POSIX style regular expression (“regex”) patterns which will result in data matching any of the given patterns as they are considered to have an implicit boolean “OR” between them.

To see the full list of statistical items which the dataplane can export run the following command in a shell:

$ sudo vpp_get_stats ls [<regex pattern>]

Note

Though the statistical items in the dataplane appear to be structured similar to a filesystem, matching does not work like shell filesystem globs, but as regular expression patterns only.

Filter patterns do not include automatic anchoring. Patterns can match any part of a given string. For example, the pattern /a will match /a, b/a, b/a/c, and so on. Regular expression anchoring can limit this as needed: To only match a string starting with /a use ^/a, to match as string ending in /a, use /a$, and to match a string exactly anchor it both ways, such as ^/a$.

Using wildcard style matching in filters must also follow regular expression syntax. The filter ab* matches the a followed by b repeated zero or more times. Thus it will match a, ab, abb, and abbb, but not ac. To match ab followed by anything else, use ab.* which will match ab plus any single character (.) repeated zero or more times.

Querying Prometheus Data¶

The URL for metrics is: http://<IP address>:9482/metrics

In the URL, <IP address> is an IP address on an interface in the appropriate namespace. For example, if Prometheus is running in the host namespace, the IP address would be from a host OS management interface on TNSR.

Note

This service is not meant to be queried by a web browser. Use a client which understands Prometheus data, or a client such as curl which will print the data returned by the service when testing.

Note

The Prometheus statistics daemon uses an AF_INET6 socket which can accept connections from both IPv4 and IPv6 clients. When viewing the daemon process, it will show tcp6, IPv6, or similar strings in the output. This is normal and does not indicate a problem.

$ sudo netstat -lntp | grep prometheus
tcp6       0      0 :::9482                 :::*                    LISTEN      2985/vpp_prometheus
$ sudo lsof +c15 -iTCP -sTCP:LISTEN | grep prometheus
vpp_prometheus_ 2985 root    3u  IPv6  39319      0t0  TCP *:9482 (LISTEN)

The dataplane tracks some metrics served by Prometheus by interface name and others by index. The Prometheus output includes entries which map the index numbers to their corresponding names. This can be used to correlate data in the monitoring system consuming the data:

# TYPE _if_names_info gauge
_if_names_info{index="0",name="local0"} 1
_if_names_info{index="1",name="WAN"} 1
_if_names_info{index="2",name="LAN"} 1
_if_names_info{index="3",name="CorpNet"} 1
_if_names_info{index="4",name="DMZ"} 1
_if_names_info{index="5",name="Guest"} 1

Protecting Prometheus¶

As mentioned in the warning at the start of this section, the Prometheus service does not have its own encryption or authentication. As such, the primary ways to protect the service are:

Isolate the service to the appropriate namespace

This is typically the host namespace, but some TNSR configurations do not have host interfaces.

Protect the network paths carrying Prometheus data

Use a directly connected secure path, such as a local management network. If the data must be transmitted remotely, encrypt the path between Prometheus and the host collecting its data, for example, with an IPsec tunnel.

Restrict access with ACLs

Limit access to the prometheus server on TCP port 9482 using access lists. When running in the host namespace, use host ACLs and when running in the dataplane namespace use standard ACLs.

Note

The default set of host ACLs denies access to the service, but take care when crafting rules to only permit access from authorized hosts or networks.

This example host ACL permits access to the Prometheus service from one IPv4 host, 198.51.100.244:

tnsr(config)# host acl prometheus
tnsr(config-host-acl)# sequence 10
tnsr(config-host-acl)# rule 10
tnsr(config-host-acl-rule)# action permit
tnsr(config-host-acl-rule)# description Allow Prometheus
tnsr(config-host-acl-rule)# match ip protocol tcp
tnsr(config-host-acl-rule)# match ip port destination 9482
tnsr(config-host-acl-rule)# match ip version 4
tnsr(config-host-acl-rule)# match ip address source 198.51.100.244/32
tnsr(config-host-acl-rule)# exit
tnsr(config-host-acl)# exit
tnsr(config)#