amqp plugin¶


Plugin name	`amqp`
Type	write
Callbacks	config, shutdown, write
Status	supported
Since	5.0
Copyright	2009 Sebastien Pahl, 2010 Florian Forster
License	MIT License
Configuration	collectd.conf(5)

The AMQP plugin transmits or receives values collected by collectd via the Advanced Message Queuing Protocol (AMQP). Data is sent to or received from a "message broker" – a daemon relaying messages. The values are encoded in either the plain-text-protocol or JSON, though only the plain-text protocol can be parsed by the plugin at the moment. Messages can be sent in either persistent (guaranteed delivery) or transient (higher efficiency but values may be lost) delivery mode.

Synopsis¶

 <Plugin "amqp">
   # Send values to an AMQP broker
   <Publish "some_name">
     Host "localhost"
     Port "5672"
     VHost "/"
     User "guest"
     Password "guest"
     Exchange "amq.fanout"
 #   ExchangeType "fanout"
 #   RoutingKey "collectd"
 #   Persistent false
 #   Format "command"
 #   StoreRates false
   </Publish>

   # Receive values from an AMQP broker (work in progress)
   <Subscribe "some_name">
     Host "localhost"
     Port "5672"
     VHost "/"
     User "guest"
     Password "guest"
     Exchange "amq.fanout"
 #   ExchangeType "fanout"
 #   Queue "queue_name"
 #   RoutingKey "collectd.#"
   </Subscribe>
 </Plugin>

this registers a writer by the name

amqp/

Routing¶

AMQP uses three components to deliver messages from a publisher to a subscriber: Exchanges are the place where publishers post their messages; Queues are the place from where subscribers receive messages. An exchange and a queue may be connected with a binding, meaning that messages may be routed from the exchange to the queue.

Which messages are sent to which queue depends on the type of the exchange and the binding.

A fanout exchange sends messages too all queues bound to it.
A direct exchange compares the routing keys of the message and the binding and sends the message to the appropriate queue only if they are identical.
A topic exchange compares the routing keys of the message and the binding and allows for wildcards (placeholders) in the binding's routing key.
A headers exchange matches header fields attached to a message to make a routing decision. Currently the AMQP plugin doesn't support the creation of header fields and can't create appropriate bindings automatically.

Exchanges¶

The AMQP plugin can automatically create an appropriate exchange after connecting to the broker. This happens if the ExchangeType configuration option has been set.

The type is given as a string which is passed to the broker. Usually this would be one of the standard strings – "direct", "fanout", "topic", or "header" – but the standard explicitly allows for implementation specific types.

Exchanges are created as non-durable and auto delete. This means the exchange won't exist after the broker was restarted and will automatically get removed when there are no more queues bound to it.

Queues¶

When subscribing, the plugin will automatically create a queue after connecting. If the Queue configuration option has been specified, this name is used, otherwise a unique name is chosen automatically by the broker.

Flags used when creating the queue are non-durable, non-exclusive, auto delete.

Bindings¶

If a Exchange configuration option is given in a Subscribe block, the (possibly newly created) queue will be bound to the given exchange. The binding is created using the routing key (if it exists) and an empty "argument table". (Setting this argument table would be required for header exchanges.)

Routing keys¶

If a routing key has been configured, the data sent to the broker will always use the given routing key. If no routing key has been configured in a Publish block, a routing key will be created from the identifier. Because a routing key consists of several fields separated by dots, all dots within the identifier will be replaced by slashes. For example:

collectd.example/collectd/org.cpu.0.cpu.idle
collectd.example/collectd/org.memory..memory.used

The routing key specified in Subscribe blocks is copied into automatically created bindings. How this routing key is evaluated depends on the type of exchange the queue is bound to. In case of a topic exchange, you can use wildcards to select only certain values. The following example requests only cpu statistics from the exchange:

collectd.*.cpu.#

Dependencies¶

rabbitmq-c

Configuration¶

Configuration options · from collectd.conf(5)

See this section in the full configuration reference

The AMQP plugin can be used to communicate with other instances of collectd or third party applications using an AMQP 0.9.1 message broker. Values are sent to or received from the broker, which handles routing, queueing and possibly filtering out messages.

Synopsis:

<Plugin "amqp">
  # Send values to an AMQP broker
  <Publish "some_name">
    Host "localhost"
    Host "fallback-amqp.example.com"
    Port "5672"
    VHost "/"
    User "guest"
    Password "guest"
    Exchange "amq.fanout"
#   ExchangeType "fanout"
#   RoutingKey "collectd"
#   Persistent false
#   ConnectionRetryDelay 0
#   Format "command"
#   StoreRates false
#   TLSEnabled false
#   TLSVerifyPeer true
#   TLSVerifyHostName true
#   TLSCACert "/path/to/ca.pem"
#   TLSClientCert "/path/to/client-cert.pem"
#   TLSClientKey "/path/to/client-key.pem"
#   GraphitePrefix "collectd."
#   GraphiteEscapeChar "_"
#   GraphiteSeparateInstances false
#   GraphiteAlwaysAppendDS false
#   GraphitePreserveSeparator false
  </Publish>

  # Receive values from an AMQP broker
  <Subscribe "some_name">
    Host "localhost"
    Port "5672"
    VHost "/"
    User "guest"
    Password "guest"
    Exchange "amq.fanout"
#   ExchangeType "fanout"
#   Queue "queue_name"
#   QueueDurable false
#   QueueAutoDelete true
#   RoutingKey "collectd.#"
#   ConnectionRetryDelay 0
#   TLSEnabled false
#   TLSVerifyPeer true
#   TLSVerifyHostName true
#   TLSCACert "/path/to/ca.pem"
#   TLSClientCert "/path/to/client-cert.pem"
#   TLSClientKey "/path/to/client-key.pem"
  </Subscribe>
</Plugin>

The plugin's configuration consists of a number of Publish and Subscribe blocks, which configure sending and receiving of values respectively. The two blocks are very similar, so unless otherwise noted, an option can be used in either block. The name given in the blocks starting tag is only used for reporting messages, but may be used to support flushing of certain Publish blocks in the future.

Host Host [Host ...]

Hostname or IP-address of the AMQP broker. Defaults to the default behavior of the underlying communications library, rabbitmq-c, which is "localhost".

If multiple hosts are specified, then a random one is chosen at each (re)connection attempt. This is useful for failover with a clustered broker.

Port Port

Service name or port number on which the AMQP broker accepts connections. This argument must be a string, even if the numeric form is used. Defaults to "5672".

VHost VHost

Name of the virtual host on the AMQP broker to use. Defaults to "/".

User User

Password Password

Credentials used to authenticate to the AMQP broker. By default "guest"/"guest" is used.

Exchange Exchange

In Publish blocks, this option specifies the exchange to send values to. By default, "amq.fanout" will be used.

In Subscribe blocks this option is optional. If given, a binding between the given exchange and the queue is created, using the routing key if configured. See the Queue and RoutingKey options below.

ExchangeType Type

If given, the plugin will try to create the configured exchange with this type after connecting. When in a Subscribe block, the queue will then be bound to this exchange.

Queue Queue (Subscribe only)

Configures the queue name to subscribe to. If no queue name was configured explicitly, a unique queue name will be created by the broker.

QueueDurable true|false (Subscribe only)

Defines if the queue subscribed to is durable (saved to persistent storage) or transient (will disappear if the AMQP broker is restarted). Defaults to "false".

This option should be used in conjunction with the Persistent option on the publish side.

QueueAutoDelete true|false (Subscribe only)

Defines if the queue subscribed to will be deleted once the last consumer unsubscribes. Defaults to "true".

RoutingKey Key

In Publish blocks, this configures the routing key to set on all outgoing messages. If not given, the routing key will be computed from the identifier of the value. The host, plugin, type and the two instances are concatenated together using dots as the separator and all containing dots replaced with slashes. For example "collectd.host/example/com.cpu.0.cpu.user". This makes it possible to receive only specific values using a "topic" exchange.

In Subscribe blocks, configures the routing key used when creating a binding between an exchange and the queue. The usual wildcards can be used to filter messages when using a "topic" exchange. If you're only interested in CPU statistics, you could use the routing key "collectd.*.cpu.#" for example.

Persistent true|false (Publish only)

Selects the delivery method to use. If set to true, the persistent mode will be used, i.e. delivery is guaranteed. If set to false (the default), the transient delivery mode will be used, i.e. messages may be lost due to high load, overflowing queues or similar issues.

ConnectionRetryDelay Delay

When the connection to the AMQP broker is lost, defines the time in seconds to wait before attempting to reconnect. Defaults to 0, which implies collectd will attempt to reconnect at each read interval (in Subscribe mode) or each time values are ready for submission (in Publish mode).

Format Command|JSON|Graphite (Publish only)

Selects the format in which messages are sent to the broker. If set to Command (the default), values are sent as PUTVAL commands which are identical to the syntax used by the Exec and UnixSock plugins. In this case, the Content-Type header field will be set to text/collectd.

If set to JSON, the values are encoded in the JavaScript Object Notation, an easy and straight forward exchange format. The Content-Type header field will be set to application/json.

If set to Graphite, values are encoded in the Graphite format, which is "<metric> <value> <timestamp>\n". The Content-Type header field will be set to text/graphite.

A subscribing client should use the Content-Type header field to determine how to decode the values. Currently, the AMQP plugin itself can only decode the Command format.

StoreRates true|false (Publish only)

Determines whether or not COUNTER, DERIVE and ABSOLUTE data sources are converted to a rate (i.e. a GAUGE value). If set to false (the default), no conversion is performed. Otherwise the conversion is performed using the internal value cache.

Please note that currently this option is only used if the Format option has been set to JSON.

GraphitePrefix (Publish and Format=Graphite only)

A prefix can be added in the metric name when outputting in the Graphite format. It's added before the Host name. Metric name will be "<prefix><host><postfix><plugin><type><name>"

GraphitePostfix (Publish and Format=Graphite only)

A postfix can be added in the metric name when outputting in the Graphite format. It's added after the Host name. Metric name will be "<prefix><host><postfix><plugin><type><name>"

GraphiteEscapeChar (Publish and Format=Graphite only)

Specify a character to replace dots (.) in the host part of the metric name. In Graphite metric name, dots are used as separators between different metric parts (host, plugin, type). Default is "_" (Underscore).

GraphiteSeparateInstances true|false

If set to true, the plugin instance and type instance will be in their own path component, for example host.cpu.0.cpu.idle. If set to false (the default), the plugin and plugin instance (and likewise the type and type instance) are put into one component, for example host.cpu-0.cpu-idle.

GraphiteAlwaysAppendDS true|false

If set to true, append the name of the Data Source (DS) to the "metric" identifier. If set to false (the default), this is only done when there is more than one DS.

GraphitePreserveSeparator false|true

If set to false (the default) the . (dot) character is replaced with GraphiteEscapeChar. Otherwise, if set to true, the . (dot) character is preserved, i.e. passed through.

TLSEnabled true|false

If set to true then connect to the broker using a TLS connection. If set to false (the default), then a plain text connection is used.

Requires rabbitmq-c >= 0.4.

TLSVerifyPeer true|false

If set to true (the default) then the server certificate chain is verified. Setting this to false will skip verification (insecure).

Requires rabbitmq-c >= 0.8.

TLSVerifyHostName true|false

If set to true (the default) then the server host name is verified. Setting this to false will skip verification (insecure).

Requires rabbitmq-c >= 0.8.

TLSCACert Path

Path to the CA cert file in PEM format.

TLSClientCert Path

Path to the client certificate in PEM format. If this is set, then TLSClientKey must be set as well.

TLSClientKey Path

Path to the client key in PEM format. If this is set, then TLSClientCert must be set as well.