curl_xml plugin¶
| Plugin name | curl_xml |
| Type | read |
| Callbacks | config, read |
| Status | supported |
| Since | 4.10 |
| Copyright | 2009,2010 Amit Gupta |
| License | GPLv2 |
| Configuration | collectd.conf(5) |
The cURL-XML plugin reads files using libcurl and parses it as Extensible Markup Language (XML). The parsed tree is then traversed according to the user's configuration, using the XML Path Language (XPath).
This plugin is a generic plugin, i.e. it cannot work without configuration, because there is no reasonable default behavior. Please read the Plugin curl_xml section of the collectd.conf(5) manual page for an in-depth description of the plugin's configuration.
Synopsis¶
- → See: Plugin-cURL-XML/Config
<Plugin "curl_xml">
<URL "<nowiki>http://localhost/stats.xml</nowiki>">
Host "my_host"
Instance "some_instance"
User "collectd"
Password "thaiNg0I"
VerifyPeer true
VerifyHost true
CACert "/path/to/ca.crt"
<xpath "table[@id=\"magic_level\"]/tr">
Type "magic_level"
#InstancePrefix "prefix-"
InstanceFrom "td[1]"
ValuesFrom "td[2]/span[@class=\"level\"]"
</xpath>
</URL>
</Plugin>
Example graphs¶
None yet. Add one now!
Dependencies¶
See also¶
- Example configurations
- cURL plugin
- cURL-JSON plugin
Configuration examples¶
This page contains some example configurations for the cURL-XML plugin. This page is meant as a cookbook, so if you have a configuration for an aspect not handled here or a daemon not present, please feel free to add anything that's useful for you.
Sun Java System Web Server 7.0¶
Sun Java System Web Server 7.0 is a web server which delivers a single, secure infrastructure for all web technologies and applications. The web server exposes the monitoring statistics via a URL which can be easily consumed by the cURL-XML plugin.
The example config collects the rate of incoming and outgoing traffic, rate of 3xx,4xx and 5xx errors and the rate of the web server requests for all the virtual servers. The config also collects the web server Java statistics like Java heap usage, current Java threads and total number of Java garbage collections for all the web server processes (if more than one).
<Plugin curl_xml>
<URL "<nowiki>http://localhost:80/stats-xml</nowiki>">
Instance "sunwebserver-inst1"
<xpath "/stats/server/virtual-server">
InstanceFrom "@id"
ValuesFrom "request-bucket/@countBytesReceived"
"request-bucket/@countBytesTransmitted"
Type "sjsws_bytes"
</xpath>
<xpath "/stats/server/virtual-server">
InstanceFrom "@id"
ValuesFrom "request-bucket/@count3xx" "request-bucket/@count4xx"
"request-bucket/@count5xx"
Type "sjsws_errors"
</xpath>
<xpath "/stats/server/virtual-server">
InstanceFrom "@id"
ValuesFrom "request-bucket/@countRequests"
Type "sjsws_requests"
</xpath>
<xpath "/stats/server/process">
InstanceFrom "@pid"
ValuesFrom "jvm/@countTotalClassesLoaded" "jvm/@countTotalClassesUnloaded"
Type "sjsws_jvm_classes"
</xpath>
<xpath "/stats/server/process">
InstanceFrom "@pid"
ValuesFrom "jvm/@countThreads" "jvm/@countTotalThreadsStarted"
Type "sjsws_jvm_threads"
</xpath>
<xpath "/stats/server/process">
InstanceFrom "@pid"
ValuesFrom "jvm/@sizeHeap"
Type "sjsws_jvm_heap"
</xpath>
<xpath "/stats/server/process">
InstanceFrom "@pid"
ValuesFrom "jvm/@countGarbageCollections"
Type "sjsws_jvm_gc"
</xpath>
</URL>
</Plugin>
Twitter statistics¶
The following configuration uses the REST API for Twitter to read the number of status updates ("Tweets") and followers. The configuration uses the screen name "foobar" which you will have to replace by the screen name you're actually interested in.
This example demonstrates how data can be extracted from text nodes using the text() syntax.
<Plugin "curl_xml">
<nowiki><URL "http://api.twitter.com/1/users/show.xml?screen_name=foobar"></nowiki>
Instance "twitter"
<XPath "/user">
Type "records"
InstancePrefix "tweets"
#InstanceFrom ""
ValuesFrom "statuses_count/text()"
</XPath>
<XPath "/user">
Type "users"
InstancePrefix "followers"
#InstanceFrom ""
ValuesFrom "followers_count/text()"
</XPath>
</URL>
</Plugin>
Because Twitter does some rate-limiting on the use that this API, you should not use a 10 second interval when querying this data! Currently (July 2010) the limit is 150 requests per hour, so querying the information every 30 seconds should be fine. For more information, see http://dev.twitter.com/pages/rate-limiting.
Fritz!Box Internet Router¶
Some models of Fritz!Box routers (e.g. 3270) allow retrieving some status information with a SOAP call. The following example shows how to call a SOAP function and parse the XML response using namespaces in order to retrieve the number of bytes sent and received by the router.
<Plugin "curl_xml">
<URL <nowiki>"http://192.168.0.1:49000/upnp/control/WANCommonIFC1"></nowiki>
Instance "router_stats"
Header <nowiki>"SoapAction: \"urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1#GetAd
donInfos\""</nowiki>
Header "Content-Type: text/xml"
Post <nowiki>"<?xml version=\"1.0\"?><s:Envelope xmlns:s=\"http://schemas.xmlsoap.org/soap/
envelope/\" s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\"><s:Body><u:G
etAddonInfos xmlns:u=\"urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1\" /></s
:Body></s:Envelope>"</nowiki>
Namespace "s" <nowiki>"http://schemas.xmlsoap.org/soap/envelope/"</nowiki>
Namespace "u" "urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1"
<XPath "/s:Envelope/s:Body/u:GetAddonInfosResponse">
Type "ipt_bytes"
InstancePrefix "tx"
ValuesFrom "NewTotalBytesSent"
</XPath>
<XPath "/s:Envelope/s:Body/u:GetAddonInfosResponse">
Type "ipt_bytes"
InstancePrefix "rx"
ValuesFrom "NewTotalBytesReceived"
</XPath>
</URL>
</Plugin>
See also¶
Configuration¶
Configuration options · from collectd.conf(5)
See this section in the full configuration reference
The curl_xml plugin uses libcurl (http://curl.haxx.se/) and libxml2 (http://xmlsoft.org/) to retrieve XML data via cURL.
<Plugin "curl_xml">
<URL "http://localhost/stats.xml">
AddressFamily "any"
Host "my_host"
#Plugin "curl_xml"
Instance "some_instance"
User "collectd"
Password "thaiNg0I"
VerifyPeer true
VerifyHost true
CACert "/path/to/ca.crt"
Header "X-Custom-Header: foobar"
Post "foo=bar"
<XPath "table[@id=\"magic_level\"]/tr">
Type "magic_level"
#InstancePrefix "prefix-"
InstanceFrom "td[1]"
#PluginInstanceFrom "td[1]"
ValuesFrom "td[2]/span[@class=\"level\"]"
</XPath>
</URL>
</Plugin>
In the Plugin block, there may be one or more URL blocks, each defining a URL to be fetched using libcurl. Within each URL block there are options which specify the connection parameters, for example authentication information, and one or more XPath blocks.
Each XPath block specifies how to get one type of information. The string argument must be a valid XPath expression which returns a list of "base elements". One value is dispatched for each "base element". The type instance and values are looked up using further XPath expressions that should be relative to the base element.
Within the URL block the following options are accepted:
- AddressFamily Type
- IP version to resolve URL to. Useful in cases when hostname in URL resolves to both IPv4 and IPv6 addresses, and you are interested in using one of them specifically. Use
ipv4to enforce IPv4,ipv6to enforce IPv6, oranyto keep the default behavior of resolving addresses to all IP versions your system allows. Iflibcurlis compiled without IPv6 support, usingipv6will result in a warning and fallback toany. IfTypecannot be parsed, a warning will be printed and the whole URL block will be ignored. - Host Name
- Use Name as the host name when submitting values. Defaults to the global host name setting.
- Plugin Plugin
- Use Plugin as the plugin name when submitting values. Defaults to 'curl_xml'.
- Instance Instance
- Use Instance as the plugin instance when submitting values. May be overridden by PluginInstanceFrom option inside XPath blocks. Defaults to an empty string (no plugin instance).
- Interval Interval
- Sets the interval (in seconds) in which the values will be collected from this URL. By default the global Interval setting will be used.
- Namespace Prefix URL
-
If an XPath expression references namespaces, they must be specified with this option. Prefix is the "namespace prefix" used in the XML document. URL is the "namespace name", an URI reference uniquely identifying the namespace. The option can be repeated to register multiple namespaces.
Examples:
- User User
- Password Password
- Digest true|false
- VerifyPeer true|false
- VerifyHost true|false
- CACert CA Cert File
- Header Header
- Post Body
- Timeout Milliseconds
- These options behave exactly equivalent to the appropriate options of the cURL plugin. Please see there for a detailed description.
- <StatisticsE<gt>
- One Statistics block can be used to specify cURL statistics to be collected for each request to the remote URL. See the section "cURL Statistics" above for details.
- <XPath XPath-expression>
-
Within each URL block, there must be one or more XPath blocks. Each XPath block specifies how to get one type of information. The string argument must be a valid XPath expression which returns a list of "base elements". One value is dispatched for each "base element".
Within the XPath block the following options are accepted:
- Type Type
- Specifies the Type used for submitting patches. This determines the number of values that are required / expected and whether the strings are parsed as signed or unsigned integer or as double values. See types.db(5) for details. This option is required.
- InstancePrefix InstancePrefix
- Prefix the type instance with InstancePrefix. The values are simply concatenated together without any separator. This option is optional.
- InstanceFrom InstanceFrom
- Specifies a XPath expression to use for determining the type instance. The XPath expression must return exactly one element. The element's value is then used as type instance, possibly prefixed with InstancePrefix (see above).
- PluginInstanceFrom PluginInstanceFrom
- Specifies a XPath expression to use for determining the plugin instance. The XPath expression must return exactly one element. The element's value is then used as plugin instance.
If the "base XPath expression" (the argument to the XPath block) returns exactly one argument, then InstanceFrom and PluginInstanceFrom may be omitted. Otherwise, at least one of InstanceFrom or PluginInstanceFrom is required.
- ValuesFrom ValuesFrom [ValuesFrom ...]
- Specifies one or more XPath expression to use for reading the values. The number of XPath expressions must match the number of data sources in the type specified with Type (see above). Each XPath expression must return exactly one element. The element's value is then parsed as a number and used as value for the appropriate value in the value list dispatched to the daemon. This option is required.