TNSR includes a Prometheus exporter which supports statistical data from the
dataplane (VPP) only. This data is typically fed into Grafana. When active,
the service listens for connections on TCP port
Configuring Grafana and its supporting systems is outside the scope of this documentation. Consult its documentation for details.
The Prometheus service on TNSR does not perform authentication or encryption. Only transmit data across trusted network paths, and do not expose the service to untrusted networks.
Data exported by Prometheus may be sensitive in nature, so protect access to the service with appropriate standard ACLs if the service is running in the dataplane namespace. Do not open access to the Prometheus port unilaterally. The default host ACL ruleset allows access to the Prometheus port.
- prometheus <namespace> enable
Enables the Prometheus Exporter service in either the
hostnamespace is more secure as limits the exposure of the service to host OS management networks.
After enabling the service, start it as described in Service Control.
- prometheus <namespace> disable
Disables the Prometheus Exporter service in the given namespace.
- prometheus <namespace> filter <regex> [<regex> […]]
- no prometheus <namespace> filter <regex>
Removes a filter.
The data exposed by Prometheus to clients can be limited by filtering. Limiting in this manner can increase performance by reducing the amount of data sent to clients and the amount that clients must process as a consequence. By default, all data in the dataplane statistics segment is exported which can be sizable on large deployments.
Prometheus filters consist of one or more POSIX style regular expression (“regex”) patterns which will result in data matching any of the given patterns as they are considered to have an implicit boolean “OR” between them.
To see the full list of statistical items which the dataplane can export run the following command in a shell:
$ sudo vpp_get_stats ls [<regex pattern>]
Though the statistical items in the dataplane appear to be structured similar to a filesystem, matching does not work like shell filesystem globs, but as regular expression patterns only.
Filter patterns do not include automatic anchoring. Patterns can match any
part of a given string. For example, the pattern
/a will match
b/a/c, and so on. Regular expression anchoring can limit this as
needed: To only match a string starting with
^/a, to match as
string ending in
/a$, and to match a string exactly anchor it
both ways, such as
Using wildcard style matching in filters must also follow regular expression
syntax. The filter
ab* matches the
a followed by
b repeated zero or
more times. Thus it will match
abbb, but not
ac. To match
ab followed by anything else, use
ab.* which will match
ab plus any single character (
.) repeated zero or more times.
Comprehensive coverage of regular expression pattern usage is beyond the scope of this documentation. There are a variety of resources on the Internet which cover regular expression usage and syntax. See regular expression page on Wikipedia for a good starting point.
When querying TNSR for Prometheus data, the items returned do not use
as a separator, but
_. For example, the dataplane item
/sys/vector_rate_per_worker will be in the Prometheus data as
_sys_vector_rate_per_worker. Always craft filter expressions to match the
dataplane version of the desired items, not the format in Prometheus
Querying Prometheus Data¶
The URL for metrics is:
In the URL,
<IP address> is an IP address on an interface in the appropriate
namespace. For example, if Prometheus is running in the
host namespace, the
IP address would be from a host OS management interface on TNSR.
This service is not meant to be queried by a web browser. Use a client
which understands Prometheus data, or a client such as
curl which will
print the data returned by the service when testing.
The Prometheus statistics daemon uses an AF_INET6 socket which can accept
connections from both IPv4 and IPv6 clients. When viewing the daemon process,
it will show
IPv6, or similar strings in the output. This is
normal and does not indicate a problem.
$ sudo netstat -lntp | grep prometheus tcp6 0 0 :::9482 :::* LISTEN 2985/vpp_prometheus $ sudo lsof +c15 -iTCP -sTCP:LISTEN | grep prometheus vpp_prometheus_ 2985 root 3u IPv6 39319 0t0 TCP *:9482 (LISTEN)
As mentioned in the warning at the start of this section, the Prometheus service does not have its own encryption or authentication. As such, the primary ways to protect the service are:
- Isolate the service to the appropriate namespace
This is typically the
hostnamespace, but some TNSR configurations do not have host interfaces.
- Protect the network paths carrying Prometheus data
Use a directly connected secure path, such as a local management network. If the data must be transmitted remotely, encrypt the path between Prometheus and the host collecting its data, for example, with an IPsec tunnel.
- Restrict access with ACLs
The default set of host ACLs denies access to the service, but take care when crafting rules to only permit access from authorized hosts or networks.
This example host ACL permits access to the Prometheus service from one IPv4 host,
tnsr(config)# host acl prometheus tnsr(config-host-acl)# sequence 10 tnsr(config-host-acl)# rule 10 tnsr(config-host-acl-rule)# action permit tnsr(config-host-acl-rule)# description Allow Prometheus tnsr(config-host-acl-rule)# match ip protocol tcp tnsr(config-host-acl-rule)# match ip port destination 9482 tnsr(config-host-acl-rule)# match ip version 4 tnsr(config-host-acl-rule)# match ip address source 198.51.100.244/32 tnsr(config-host-acl-rule)# exit tnsr(config-host-acl)# exit tnsr(config)#