write_http plugin¶


Plugin name	`write_http`
Type	write
Callbacks	config, flush, write
Status	supported
Since	4.8
Copyright	2009 Paul Sadauskas, 2009 Doug MacEachern, 2007–2014 Florian octo Forster
License	GPLv2
Configuration	collectd.conf(5)

The Write HTTP plugin sends the values collected by collectd to a web-server using HTTP POST requests. The data is formatted as PUTVAL commands, see plain-text-protocol.

In order to improve performance the plugin fills a 4 kByte buffer as full as possible and sends multiple lines at once.

Synopsis¶

 <Plugin "write_http">
   <Node "example">
     URL "<nowiki>http://example.com/post-collectd</nowiki>"
     User "collectd"
     Password "weCh3ik0"
   </Node>
 </Plugin>

Example data¶

The data transmitted to the web-server has the following form:

Command format

 PUTVAL leeloo.lan.home.verplant.org/disk-sda/disk_octets interval=10 1251533299:197141504:175136768
 PUTVAL leeloo.lan.home.verplant.org/disk-sda/disk_ops interval=10 1251533299:10765:12858
 PUTVAL leeloo.lan.home.verplant.org/disk-sda/disk_time interval=10 1251533299:5:140
 PUTVAL leeloo.lan.home.verplant.org/disk-sda/disk_merged interval=10 1251533299:4658:29899
 ⋮

JSON format

 [
   {
     "values": [197141504, 175136768],
     "dstypes": ["counter", "counter"],
     "dsnames": ["read", "write"],
     "time": 1251533299,
     "interval": 10,
     "host": "leeloo.lan.home.verplant.org",
     "plugin": "disk",
     "plugin_instance": "sda",
     "type": "disk_octets",
     "type_instance": ""
   },
   …
 ]

RestMQ¶

You can use RestMQ with the collectd "entry point" to consume the data sent by the Write HTTP plugin. RestMQ is a message queue based on Redis which provides a RESTful interface. It includes a special "entry point" which parses the PUTVAL command data and emits the data in a specialized JSON format, it also accepts the JSON produced by the collectd daemon.

Plain Text Protocol Example¶

You need to set the Format option to "Command" (the default) in order to work with this module:

Synopsis

 <Plugin "write_http">
   <Node "restmq_example">
     URL "<nowiki>http://restmq.example.com/collectd/data</nowiki>"
     Format "Command"
     #User "collectd"
     #Password "weCh3ik0"
   </Node>
 </Plugin>

Example JSON

 [{
   "host": "collectd_restmq_server",
   "type_instance": "df-boot",
   "plugin_instance": "df",
   "value": "1288548767:116497408.000000:391246848.000000",
   "interval": "10"
 }]

JSON Example¶

You need to set the Format option to "JSON" in order to work with this module:

Synopsis

 <Plugin "write_http">
   <Node "restmq_example">
     URL "<nowiki>http://restmq.example.com/collectd/data</nowiki>"
     Format "JSON"
     #User "collectd"
     #Password "weCh3ik0"
   </Node>
 </Plugin>

Example JSON

 [{
   "values": [1.16497e+08,3.91247e+08],
   "time":1288638055,
   "interval":10,
   "host":"collectd_restmq_server",
   "plugin":"df",
   "plugin_instance":"",
   "type":"df",
   "type_instance":"boot"}
 }]

Chains Example¶

A common requirement is the ability to direct selected metrics toward the 'write_http' plugin, while continuing to process other metrics via another write plugin:

Synopsis

 LoadPlugin write_http
 <Plugin "write_http">
   <Node "example_node">
     URL "<nowiki>http://example.com/post-collectd</nowiki>"
   </Node>
 </Plugin>
 PostCacheChain "PostCache"
 <Chain "PostCache">
   <Rule "snmp">
     <Match "regex">
       Plugin "^snmp$"
     </Match>
     <Target "write">
       Plugin "write_http/example_node"
     </Target>
     Target "stop"
   </Rule>
   # Default target
   <Target "write">
     Plugin "rrdtool"
   </Target>
 </Chain>

Weblinks

RestMQ's official website
RestMQ on Github
README.collectd

Dependencies¶

libcurl

Configuration¶

Configuration options · from collectd.conf(5)

See this section in the full configuration reference

This output plugin submits values to an HTTP server using POST requests and encoding metrics with JSON or using the PUTVAL command described in collectd-unixsock(5).

Synopsis:

<Plugin "write_http">
  <Node "example">
    URL "http://example.com/post-collectd"
    User "collectd"
    Password "weCh3ik0"
    Format JSON
  </Node>
</Plugin>

The plugin can send values to multiple HTTP servers by specifying one <Node Name> block for each server. Within each Node block, the following options are available:

URL URL

URL to which the values are submitted to. Mandatory.

User Username

Optional user name needed for authentication.

Password Password

Optional password needed for authentication.

VerifyPeer true|false

Enable or disable peer SSL certificate verification. See http://curl.haxx.se/docs/sslcerts.html for details. Enabled by default.

VerifyHost true|false

Enable or disable peer host name verification. If enabled, the plugin checks if the Common Name or a Subject Alternate Name field of the SSL certificate matches the host name provided by the URL option. If this identity check fails, the connection is aborted. Obviously, only works when connecting to a SSL enabled server. Enabled by default.

CACert File

File that holds one or more SSL certificates. If you want to use HTTPS you will possibly need this option. What CA certificates come bundled with libcurl and are checked by default depends on the distribution you use.

CAPath Directory

Directory holding one or more CA certificate files. You can use this if for some reason all the needed CA certificates aren't in the same file and can't be pointed to using the CACert option. Requires libcurl to be built against OpenSSL.

ClientKey File

File that holds the private key in PEM format to be used for certificate-based authentication.

ClientCert File

File that holds the SSL certificate to be used for certificate-based authentication.

ClientKeyPass Password

Password required to load the private key in ClientKey.

Header Header

A HTTP header to add to the request. Multiple headers are added if this option is specified more than once. Example:

Header "X-Custom-Header: custom_value"

Define which SSL protocol version must be used. By default libcurl will attempt to figure out the remote SSL protocol version. See curl_easy_setopt(3) for more details.

Format Command|JSON|KAIROSDB

Format of the output to generate. If set to Command, will create output that is understood by the Exec and UnixSock plugins. When set to JSON, will create output in the JavaScript Object Notation (JSON). When set to KAIROSDB , will create output in the KairosDB format.

Defaults to Command.

Attribute String String

Only available for the KAIROSDB output format.

Consider the two given strings to be the key and value of an additional tag for each metric being sent out.

You can add multiple Attribute.

TTL Int

Only available for the KAIROSDB output format.

Sets the Cassandra ttl for the data points.

Please refer to http://kairosdb.github.io/docs/build/html/restapi/AddDataPoints.html?highlight=ttl

Prefix String

Only available for the KAIROSDB output format.

Sets the metrics prefix string. Defaults to collectd.

Metrics true|false

Controls whether metrics are POSTed to this location. Defaults to true.

Notifications false|true

Controls whether notifications are POSTed to this location. Defaults to false.

StoreRates true|false

If set to true, convert counter values to rates. If set to false (the default) counter values are stored as is, i.e. as an increasing integer number.

BufferSize Bytes

Sets the send buffer size to Bytes. By increasing this buffer, less HTTP requests will be generated, but more metrics will be batched / metrics are cached for longer before being sent, introducing additional delay until they are available on the server side. Bytes must be at least 1024 and cannot exceed the size of an int, i.e. 2E<nbsp>GByte. Defaults to 4096.

LowSpeedLimit Bytes per Second

Sets the minimal transfer rate in Bytes per Second below which the connection with the HTTP server will be considered too slow and aborted. All the data submitted over this connection will probably be lost. Defaults to 0, which means no minimum transfer rate is enforced.

Timeout Timeout

Sets the maximum time in milliseconds given for HTTP POST operations to complete. When this limit is reached, the POST operation will be aborted, and all the data in the current send buffer will probably be lost. Defaults to 0, which means the connection never times out.

LogHttpError false|true

Enables printing of HTTP error code to log. Turned off by default.

<Statistics Name>

One Statistics block can be used to specify cURL statistics to be collected for each request to the remote URL. See the section "cURL Statistics" above for details.

The write_http plugin regularly submits the collected values to the HTTP server. How frequently this happens depends on how much data you are collecting and the size of BufferSize. The optimal value to set Timeout to is slightly below this interval, which you can estimate by monitoring the network traffic between collectd and the HTTP server.

write_http plugin¶

Synopsis¶

Example data¶

RestMQ¶

Plain Text Protocol Example¶

JSON Example¶

Chains Example¶

Dependencies¶

See also¶

Configuration¶