varnish plugin¶
| Plugin name | varnish |
| Type | read |
| Callbacks | config, init, read |
| Status | supported |
| Since | 5.0 |
| Copyright | 2010 Jérôme Renard, 2010 Marc Fournier, 2010 Florian Forster |
| License | GPLv2 |
| Configuration | collectd.conf(5) |
Description¶
The Varnish plugin collects information about Varnish, an HTTP accelerator.
Synopsis¶
<Plugin "varnish">
<Instance>
CollectCache true
CollectConnections true
CollectBackend true
CollectSHM true
CollectESI false
CollectFetch false
CollectHCB false
CollectSMA false
CollectSMS false
CollectSM false
CollectTotals false
CollectWorkers false
</Instance>
</Plugin>
Example graphs¶
Available statistics¶
| Setting | Description | Default |
|---|---|---|
CollectBackend |
Back-end connection statistics, such as successful, reused, and closed connections. | enabled |
CollectCache |
Cache hits and misses. | enabled |
CollectConnections |
Number of client connections received, accepted and dropped. | enabled |
CollectESI |
Edge Side Includes (ESI) parse statistics. | disabled |
CollectFetch |
Statistics about fetches (HTTP requests sent to the backend). | disabled |
CollectHCB |
Inserts and look-ups in the crit bit tree based hash. Look-ups are divided into locked and unlocked look-ups. | disabled |
CollectSHM |
Statistics about the shared memory log, a memory region to store log messages which is flushed to disk when full. | enabled |
CollectSM |
file (memory mapped file) storage statistics. | disabled |
CollectSMA |
malloc or umem (umem_alloc(3MALLOC) based) storage statistics. The umem storage component is Solaris specific. | disabled |
CollectSMS |
synth (synthetic content) storage statistics. This storage component is used internally only. | disabled |
CollectTotals |
Collects overview counters, such as the number of sessions created, the number of requests and bytes transferred. | disabled |
CollectWorkers |
Collect statistics about worker threads. | disabled |
Information for Varnish power users¶
Below is the list of available statistics and whether or not they can be collected with the Varnish plugin:
| Field name | Description | Collected (Yes/No) | Collectd option |
|---|---|---|---|
uptime |
Child uptime | No | n/a |
client_conn |
Client connections accepted | Yes | CollectConnections |
client_drop |
Connection dropped, no sess | Yes | |
client_req |
Client requests received | Yes | |
cache_hit |
Cache hits | Yes | CollectCache |
cache_hitpass |
Cache hits for pass | Yes | |
cache_miss |
Cache misses | Yes | |
backend_conn |
Backend conn. success | Yes | CollectBackend |
backend_unhealthy |
Backend conn. not attempted | Yes | |
backend_busy |
Backend conn. too many | Yes | |
backend_fail |
Backend conn. failures | Yes | |
backend_reuse |
Backend conn. reuses | Yes | |
backend_toolate |
Backend conn. was closed | Yes | |
backend_recycle |
Backend conn. recycles | Yes | |
backend_unused |
Backend conn. unused | Yes | |
fetch_head |
Fetch head | Yes | CollectFetch |
fetch_length |
Fetch with Length | Yes | |
fetch_chunked |
Fetch chunked | Yes | |
fetch_eof |
Fetch EOF | Yes | |
fetch_bad |
Fetch had bad headers | Yes | |
fetch_close |
Fetch wanted close | Yes | |
fetch_oldhttp |
Fetch pre HTTP/1.1 closed | Yes | |
fetch_zero |
Fetch zero len | Yes | |
fetch_failed |
Fetch failed | Yes | |
n_sess_mem |
N struct sess_mem | No | n/a |
n_sess |
N struct sess | No | |
n_object |
N struct object | No | |
n_vampireobject |
N unresurrected objects | No | |
n_objectcore |
N struct objectcore | No | |
n_objecthead |
N struct objecthead | No | |
n_smf |
N struct smf | No | |
n_smf_frag |
N small free smf | No | |
n_smf_large |
N large free smf | No | |
n_vbe_conn |
N struct vbe_conn | No | |
n_wrk |
N worker threads | Yes | CollectWorkers |
n_wrk_create |
N worker threads created | Yes | |
n_wrk_failed |
N worker threads not created | Yes | |
n_wrk_max |
N worker threads limited | Yes | |
n_wrk_queue |
N queued work requests | Yes | |
n_wrk_overflow |
N overflowed work requests | Yes | |
n_wrk_drop |
N dropped work requests | Yes | |
n_backend |
N backends | No | n/a |
n_expired |
N expired objects | No | |
n_lru_nuked |
N LRU nuked objects | No | |
n_lru_saved |
N LRU saved objects | No | |
n_lru_moved |
N LRU moved objects | No | |
n_deathrow |
N objects on deathrow | No | |
losthdr |
HTTP header overflows | No | |
n_objsendfile |
Objects sent with sendfile | No | |
n_objwrite |
Objects sent with write | No | |
n_objoverflow |
Objects overflowing workspace | No | |
s_sess |
Total Sessions | Yes | CollectTotals |
s_req |
Total Requests | Yes | |
s_pipe |
Total pipe | Yes | |
s_pass |
Total pass | Yes | |
s_fetch |
Total fetch | Yes | |
s_hdrbytes |
Total header bytes | Yes | |
s_bodybytes |
Total body bytes | Yes | |
sess_closed |
Session Closed | No | n/a |
sess_pipeline |
Session Pipeline | No | |
sess_readahead |
Session Read Ahead | No | |
sess_linger |
Session Linger | No | |
sess_herd |
Session herd | No | |
shm_records |
SHM records | Yes | CollectSHM |
shm_writes |
SHM writes | Yes | |
shm_flushes |
SHM flushes due to overflow | Yes | |
shm_cont |
SHM MTX contention | Yes | |
shm_cycles |
SHM cycles through buffer | Yes | |
sm_nreq |
allocator requests | Yes | CollectSM |
sm_nobj |
outstanding allocations | Yes | |
sm_balloc |
bytes allocated | Yes | |
sm_bfree |
bytes free | Yes | |
sma_nreq |
SMA allocator requests | Yes | CollectSMA |
sma_nobj |
SMA outstanding allocations | Yes | |
sma_nbytes |
SMA outstanding bytes | Yes | |
sma_balloc |
SMA bytes allocated | Yes | |
sma_bfree |
SMA bytes free | Yes | |
sms_nreq |
SMS allocator requests | Yes | CollectSMS |
sms_nobj |
SMS outstanding allocations | Yes | |
sms_nbytes |
SMS outstanding bytes | Yes | |
sms_balloc |
SMS bytes allocated | Yes | |
sms_bfree |
SMS bytes freed | Yes | |
backend_req |
Backend requests made | No | n/a |
n_vcl |
N vcl total | No | |
n_vcl_avail |
N vcl available | No | |
n_vcl_discard |
N vcl discarded | No | |
n_purge |
N total active purges | No | |
n_purge_add |
N new purges added | No | |
n_purge_retire |
N old purges deleted | No | |
n_purge_obj_test |
N objects tested | No | |
n_purge_re_test |
N regexps tested against | No | |
n_purge_dups |
N duplicate purges removed | No | |
hcb_nolock |
HCB Lookups without lock | Yes | CollectHCB |
hcb_lock |
HCB Lookups with lock | Yes | |
hcb_insert |
HCB Inserts | Yes | |
esi_parse |
Objects ESI parsed (unlock) | Yes | CollectESI |
esi_errors |
ESI parse errors (unlock) | Yes | |
Caveats¶
Unresolved symbols in libvarnishapi¶
(This is fixed as of release 2.1.4 in official varnish-cache.org repository, tested on Debian)
Version 2.1.2 of Varnish and likely other versions had unresolved symbols in the varnishapi library used by collectd. This led to the configure script being unable to link a test program and thus considering the library as unavailable.
There's a thread on the mailing-list discussing these problems starting at: http://mailman.verplant.org/pipermail/collectd/2010-June/003876.html
Without going in the details, check that the commands below return the same result on you system:
$ pkg-config --libs varnishapi
-lvarnishapi -lvarnish -lvarnishcompat
$ ldconfig -p | egrep "libvarnish.*.so$"
libvarnishcompat.so (libc6,x86-64) => /usr/lib/libvarnishcompat.so
libvarnishapi.so (libc6,x86-64) => /usr/lib/libvarnishapi.so
libvarnish.so (libc6,x86-64) => /usr/lib/libvarnish.so
If not, the details of how to work around the problem are found here.
Compatibility with Varnish 3.0¶
The actual plugin is compatible with Varnish 3.0 starting with collectd version-5.1.
Previous versions were not compatible, because the function used to fetch statistics (VSL_OpenStats()) is no longer available in the Varnish 3.0 API (BC break).
Compatibility with Varnish 4.0¶
The actual plugin is compatible with Varnish 4.0 starting with collectd version-5.5.
Dependencies¶
- libvarnishapi
See also¶
- http://varnish-cache.org/ Homepage of the Varnish project
- http://varnish-cache.org/wiki/StatsExplained Article "Stats Explained" in the Varnish wiki. (Incomplete as of June 2010.)
Configuration¶
Configuration options · from collectd.conf(5)
See this section in the full configuration reference
The varnish plugin collects information about Varnish, an HTTP accelerator. It collects a subset of the values displayed by varnishstat(1), and organizes them in categories which can be enabled or disabled. Currently only metrics shown in varnishstat(1)'s MAIN section are collected. The exact meaning of each metric can be found in varnish-counters(7).
Synopsis:
<Plugin "varnish">
<Instance "example">
CollectBackend true
CollectBan false
CollectCache true
CollectConnections true
CollectDirectorDNS false
CollectESI false
CollectFetch false
CollectHCB false
CollectObjects false
CollectPurge false
CollectSession false
CollectSHM true
CollectSMA false
CollectSMS false
CollectSM false
CollectStruct false
CollectTotals false
CollectUptime false
CollectVCL false
CollectVSM false
CollectWorkers false
CollectLock false
CollectMempool false
CollectManagement false
CollectSMF false
CollectVBE false
CollectMSE false
</Instance>
</Plugin>
The configuration consists of one or more <Instance Name> blocks. Name is the parameter passed to "varnishd -n". If left empty, it will collectd statistics from the default "varnishd" instance (this should work fine in most cases).
Inside each <Instance> blocks, the following options are recognized:
- CollectBackend true|false
- Back-end connection statistics, such as successful, reused, and closed connections. True by default.
- CollectBan true|false
- Statistics about ban operations, such as number of bans added, retired, and number of objects tested against ban operations. Only available with Varnish 3.x and above. False by default.
- CollectCache true|false
- Cache hits and misses. True by default.
- CollectConnections true|false
- Number of client connections received, accepted and dropped. True by default.
- CollectDirectorDNS true|false
- DNS director lookup cache statistics. Only available with Varnish 3.x. False by default.
- CollectESI true|false
- Edge Side Includes (ESI) parse statistics. False by default.
- CollectFetch true|false
- Statistics about fetches (HTTP requests sent to the backend). False by default.
- CollectHCB true|false
- Inserts and look-ups in the crit bit tree based hash. Look-ups are divided into locked and unlocked look-ups. False by default.
- CollectObjects true|false
- Statistics on cached objects: number of objects expired, nuked (prematurely expired), saved, moved, etc. False by default.
- CollectPurge true|false
- Statistics about purge operations, such as number of purges added, retired, and number of objects tested against purge operations. Only available with Varnish 2.x. False by default.
- CollectSession true|false
- Client session statistics. Number of past and current sessions, session herd and linger counters, etc. False by default. Note that if using Varnish 4.x, some metrics found in the Connections and Threads sections with previous versions of Varnish have been moved here.
- CollectSHM true|false
- Statistics about the shared memory log, a memory region to store log messages which is flushed to disk when full. True by default.
- CollectSMA true|false
- malloc or umem (umem_alloc(3MALLOC) based) storage statistics. The umem storage component is Solaris specific. Note: SMA, SMF and MSE share counters, enable only the one used by the Varnish instance. Available with Varnish 2.x, varnish 4.x and above (Not available in varnish 3.x). False by default.
- CollectSMS true|false
- synth (synthetic content) storage statistics. This storage component is used internally only. False by default.
- CollectSM true|false
- file (memory mapped file) storage statistics. Only available with Varnish 2.x, in varnish 4.x and above use CollectSMF. False by default.
- CollectStruct true|false
- Current varnish internal state statistics. Number of current sessions, objects in cache store, open connections to backends (with Varnish 2.x), etc. False by default.
- CollectTotals true|false
- Collects overview counters, such as the number of sessions created, the number of requests and bytes transferred. False by default.
- CollectUptime true|false
- Varnish uptime. Only available with Varnish 3.x and above. False by default.
- CollectVCL true|false
- Number of total (available + discarded) VCL (config files). False by default.
- CollectVSM true|false
- Collect statistics about Varnish's shared memory usage (used by the logging and statistics subsystems). Only available with Varnish 4.x. False by default.
- CollectWorkers true|false
- Collect statistics about worker threads. False by default.
- CollectVBE true|false
- Backend counters. Only available with Varnish 4.x and above. False by default.
- CollectSMF true|false
- file (memory mapped file) storage statistics. Only available with Varnish 4.x and above. Note: SMA, SMF and MSE share counters, enable only the one used by the Varnish instance. Used to be called SM in Varnish 2.x. False by default.
- CollectManagement true|false
- Management process counters. Only available with Varnish 4.x and above. False by default.
- CollectLock true|false
- Lock counters. Only available with Varnish 4.x and above. False by default.
- CollectMempool true|false
- Memory pool counters. Only available with Varnish 4.x and above. False by default.
- CollectMSE true|false
- Varnish Massive Storage Engine 2.0 (MSE2) is an improved storage backend for Varnish, replacing the traditional malloc and file storages. Only available with Varnish-Plus 4.x and above. Note: SMA, SMF and MSE share counters, enable only the one used by the Varnish instance. False by default.
- CollectGOTO true|false
- vmod-goto counters. Only available with Varnish Plus 6.x. False by default.