write_riemann plugin¶


Plugin name	`write_riemann`
Type	write
Callbacks	config, notification, write
Status	supported
Since	5.3
Copyright	2012,2013 Pierre-Yves Ritschard, 2013 Florian Forster
License	MIT License
Configuration	collectd.conf(5)

The Write Riemann plugin stores values in Riemann, a stream processing and monitoring system.

Riemann is a stream processing system that can be used for querying, manipulating and visualizing value streams. This makes Riemann a powerful and useful companion for collectd because it allows more sophisticated alerting than is possible with collectd's built-in thresholds.

The Write Riemann plugin uses UDP packets or a TCP connection to send data to Riemann. The data is binary encoded using Protocol Buffers (protobuf), see section #Example-data below.

You can configure multiple Node blocks to send data to multiple instances of Riemann. You can use the name specified in the block to send data selectively, using filters. For example, if you configure the connection to an instance of Riemann with

 <Node "foobar">
   # ...
 </Node>

you can write to this instance only using the block syntax of the Write target:

 <Target "write">
   Plugin "write_riemann/foobar"
 </Target>

Synopsis¶

 <Plugin "write_riemann">
  <Node "example">
    Host "localhost"
    Port "5555"
    Protocol UDP
    StoreRates true
    AlwaysAppendDS false
  </Node>
  Tag "foobar"
 </Plugin>

This will register a writer with the name write_riemann/example.

Example data¶

Data is sent as Msg objects, encoded in a protobuf. You can find the definition of the message types in src/riemann.proto.

 events {
   time: 1358875608
   service: "cpu-0/cpu-idle"
   host: "server.example.com"
   attribute {
     key: "plugin"
     value: "cpu"
   }
   attribute {
     key: "plugin_instance"
     value: "0"
   }
   attribute {
     key: "type"
     value: "cpu"
   }
   attribute {
     key: "type_instance"
     value: "idle"
   }
   attribute {
     key: "ds_type"
     value: "derive:rate"
   }
   attribute {
     key: "ds_name"
     value: "value"
   }
   attribute {
     key: "ds_index"
     value: "0"
   }
   ttl: 20
   metric_d: 66.600990955859146
 }

Dependencies¶

The <google/protobuf-c/protobuf-c.h> header file.

Caveats¶

Precision¶

Currently, Riemann provides a precision of one second. We recommend to use at least a two-second interval when sending data to Riemann, to avoid "duplicate" metrics when the time is rounded to the nearest second.

Configuration¶

Configuration options · from collectd.conf(5)

See this section in the full configuration reference

The write_riemann plugin will send values to Riemann, a powerful stream aggregation and monitoring system. The plugin sends Protobuf encoded data to Riemann using UDP packets.

Synopsis:

<Plugin "write_riemann">
  <Node "example">
    Host "localhost"
    Port "5555"
    Protocol UDP
    StoreRates true
    AlwaysAppendDS false
    TTLFactor 2.0
  </Node>
  Tag "foobar"
  Attribute "foo" "bar"
</Plugin>

The following options are understood by the write_riemann plugin:

<Node Name>

The plugin's configuration consists of one or more Node blocks. Each block is given a unique Name and specifies one connection to an instance of Riemann. Indise the Node block, the following per-connection options are understood:

Host Address

Hostname or address to connect to. Defaults to localhost.

Port Service

Service name or port number to connect to. Defaults to 5555.

Protocol UDP|TCP|TLS

Specify the protocol to use when communicating with Riemann. Defaults to TCP.

TLSCertFile Path

When using the TLS protocol, path to a PEM certificate to present to remote host.

TLSCAFile Path

When using the TLS protocol, path to a PEM CA certificate to use to validate the remote hosts's identity.

TLSKeyFile Path

When using the TLS protocol, path to a PEM private key associated with the certificate defined by TLSCertFile.

Batch true|false

If set to true and Protocol is set to TCP, events will be batched in memory and flushed at regular intervals or when BatchMaxSize is exceeded.

Notifications are not batched and sent as soon as possible.

When enabled, it can occur that events get processed by the Riemann server close to or after their expiration time. Tune the TTLFactor and BatchMaxSize settings according to the amount of values collected, if this is an issue.

Defaults to true

BatchMaxSize size

Maximum payload size for a riemann packet. Defaults to 8192

BatchFlushTimeout seconds

Maximum amount of seconds to wait in between to batch flushes. No timeout by default.

StoreRates true|false

If set to true (the default), convert counter values to rates. If set to false counter values are stored as is, i.e. as an increasing integer number.

This will be reflected in the ds_type tag: If StoreRates is enabled, converted values will have "rate" appended to the data source type, e.g. ds_type:derive:rate.

AlwaysAppendDS false|true

If set to true, append the name of the Data Source (DS) to the "service", i.e. the field that, together with the "host" field, uniquely identifies a metric in Riemann. If set to false (the default), this is only done when there is more than one DS.

TTLFactor Factor

Riemann events have a Time to Live (TTL) which specifies how long each event is considered active. collectd populates this field based on the metrics interval setting. This setting controls the factor with which the interval is multiplied to set the TTL. The default value is 2.0. Unless you know exactly what you're doing, you should only increase this setting from its default value.

Notifications false|true

If set to true, create riemann events for notifications. This is true by default. When processing thresholds from write_riemann, it might prove useful to avoid getting notification events.

CheckThresholds false|true

If set to true, attach state to events based on thresholds defined in the Threshold plugin. Defaults to false.

EventServicePrefix String

Add the given string as a prefix to the event service name. If EventServicePrefix not set or set to an empty string (""), no prefix will be used.

Tag String

Add the given string as an additional tag to the metric being sent to Riemann.

Attribute String String

Consider the two given strings to be the key and value of an additional attribute for each metric being sent out to Riemann.