Varnish Cache is a web application accelerator (also known as a caching HTTP reverse proxy) that is installed and configured in front of any HTTP and takes care of caching its contents. Varnish Cache is really fast, typically speeding up delivery with a factor of 300 - 1000x depending on the architecture.
Apart from its performance, another of its key features is the flexibility of its configuration language, VCL. VCL enables you to write policies on how incoming requests should be handled. In such a policy you can decide what content you want to serve, from where you want to get the content and how the request or response should be altered.
Which requirements does it have?
Varnish requires a working compiler (such as gcc) to compile its configuration file, which is then dynamically linked into the server process.
How to enable Varnish?
Varnish is disabled by default so it cannot be started using the ctlscripts. Enabling it is as simple as renaming its script from ctl.sh.disabled to ctl.sh:
cd <installdir>/ mv scripts/ctl.sh.disabled scripts/ctl.sh
In case you are using the Virtual Appliance or the Amazon Cloud Image, the installation directory is /opt/bitnami and you have to use "sudo" to move the file.
How to start Varnish?
Varnish is included with a default port configured as the first free port after the selected Apache port. Thant should be enough to do a preliminary test to check all is in place. You just need to execute:
$ ./ctlscript.sh start varnish SMF.s0: filename: /opt/bitnami/varnish/var/rubystack-2.3.14-9/varnish_storage.bin size 1024 MB. /opt/bitnami/varnish/scripts/ctl.sh : varnish started at port 81
And ensure Apache is running:
$ ./ctlscript.sh start apache Syntax OK /opt/bitnami/apache2/scripts/ctl.sh : httpd started at port 80
Now you should be able to access the index page in both ports, 80 and 81. Accessing the server through the 80 port will behave as if Varnish were not enabled, retrieving all the data from the server. Accessing the 81 port however will use Varnish as a reverse proxy, serving cached contents and asking to Apache for uncached ones.
How to check if Varnish is working?
You can check what Varnish is doing under the hood using the `varnishlog` command. To indicate which instance of Varnish you are interested in, you must specify the Varnish working directory, which is by default `${installdir}/varnish/var/varnish/`:
varnishlog
0 CLI - Rd ping
0 CLI - Wr 200 19 PONG 1340840690 1.0
0 CLI - Rd ping
0 CLI - Wr 200 19 PONG 1340840693 1.0
0 CLI - Rd ping
0 CLI - Wr 200 19 PONG 1340840696 1.0
If you visit your server url though the configured Varnish port (81 in our example) you will see a more interesting output:
15 Hash c /favicon.ico 15 Hash c 75.101.208.108 15 VCL_return c hash 15 VCL_call c pass pass 15 Backend c 14 default default 15 TTL c 1976586397 RFC 120 -1 -1 1340840847 0 1340840847 0 0 15 VCL_call c fetch 15 TTL c 1976586397 VCL 120 -1 -1 1340840847 -0 .... 15 TxResponse c OK 15 TxHeader c Server: Apache 15 TxHeader c X-Powered-By: PHP/5.3.13 15 TxHeader c Content-Type: image/vnd.microsoft.icon 15 TxHeader c Content-Length: 0 15 TxHeader c Accept-Ranges: bytes 15 TxHeader c Date: Wed, 27 Jun 2012 23:47:27 GMT 15 TxHeader c X-Varnish: 1976586397
To get a clearer idea of what is happening, you can use `varnishstat` instead:
varnishstat
Hitrate ratio: 1 1 1
Hitrate avg: 0.0000 0.0000 0.0000
35 0.00 0.02 client_conn - Client connections accepted
23 0.00 0.01 client_req - Client requests received
8 0.00 0.00 cache_miss - Cache misses
The command shows much more information but the clearer indication of if it is working is by checking the Hitrate ratio (how often Varnish finds the contents in its cache) and the cache_misses (how much times it failed and had to contact Apache). If you navigate your site for a while, you may find something like the below:
0+00:38:06 /opt/bitnami/varnish/var/varnish
Hitrate ratio: 10 62 62
Hitrate avg: 0.9990 0.9677 0.9677
7393 0.00 3.23 client_conn - Client connections accepted
7380 0.00 3.23 client_req - Client requests received
7354 0.00 3.22 cache_hit - Cache hits
How to change the Varnish and Apache ports?
cd <installdir> ./ctlscript.sh stop apache ./ctlscript.sh stop varnish
- Move Apache to a different port so 80 is available:
... # Change this to Listen on specific IP addresses as shown below to # prevent Apache from glomming onto all bound IP addresses. # #Listen 12.34.56.78:80 Listen 80 ...
... # Change this to Listen on specific IP addresses as shown below to # prevent Apache from glomming onto all bound IP addresses. # #Listen 12.34.56.78:80 Listen 81 ...
#! /bin/sh ... VARNISH_PORT=81 ...
- Configure Varnish to use the old Apache port (80) and specify the new port in which Apache will be listening in the configuration file (81).
#! /bin/sh ... VARNISH_PORT=81 ... VARNISH_CONFIG_FILE=/opt/bitnami/varnish/etc/varnish/default.vcl ...
So we change it to 80:
#! /bin/sh ... VARNISH_PORT=80 ... VARNISH_CONFIG_FILE=/opt/bitnami/varnish/etc/varnish/default.vcl ...
backend default {
.host = "127.0.0.1";
.port = "80";
} And we change the port from 80 to the new Apache port, 81:
backend default {
.host = "127.0.0.1";
.port = "81";
} After restarting, Apache uncached should be available at port 81 and Varnish at 80 as revese proxy for Apache:
cd <installdir> ./ctlscript.sh restart
How to change the configuration file?
Varnish is installed with a default configuration file, agnostic to the web application being cached. Using this configuration file, although achieving high performance, could lead to some contents not properly refreshed in the cache so your users would get an outdated version of your site.
The solution is to use a particularized VCL configuration file. There are multiple sources on the Internet that provides customized configuration files for different applications. A good source is the Varnish examples page. In this example, we will change our default.vcl configuration file to a WordPress-specific one (you can find the source file here).
The file requires a little modification to register the port in which your Apache server will be running. This port can be read either from the Apache configuration, looking for the "Listen" directive (in <installdir>/apache2/conf/httpd.conf):
... # Change this to Listen on specific IP addresses as shown below to # prevent Apache from glomming onto all bound IP addresses. # #Listen 12.34.56.78:80 Listen 80 ...
Or executing in a console:
$ egrep '^Listen ' <installdir>/apache2/conf/httpd.conf Listen 80
With this value (80), we edit the dowloaded wordpres.vcl and look for the section below, usually at the top of the file :
backend default {
.host = "127.0.0.1";
.port = "8080";
}
And we change the port:
backend default {
.host = "127.0.0.1";
.port = "81";
}
In the case of the BitNami stacks, Varnish is installed in the same server as Apache so the .host can be configured as `127.0.0.1` but you could also use Varnish to cache a remote server. In that case you should also provide the host IP.
After performing this modification, we copy the file to the varnish directory:
cp /path/to/the/wordpress.vcl <installdir>/varnish/etc/varnish/
And then change the varnish ctl scripts to load the appropriate file. To do that, we first ensure Varnish is stopped:
cd <installdir> ./ctlscript.sh stop varnish
And then edit the file `<installdir>/varnish/scripts/ctl.sh` and change the `VARNISH_CONFIG_FILE` variable to point to the new file:
#! /bin/sh ... VARNISH_CONFIG_FILE=/opt/bitnami/varnish/etc/varnish/wordpress.vcl ...
And now we restart Varnish (and apache if needed):
./ctlscript.sh start varnish ./ctlscript.sh start apache
How to check the performance gain using Varnish?
Although the real test will need real users interacting with your application, there are multiple solutions to perform rough load tests. In the below graphics we used blitz.io, a service that provides a free test with one minute of duration from 250 concurrent users.
The selected application to test was WordPress Stack, running in an Amazon micro instance:
You can see how around 25 users the response time starts growing really fast, and how it start failing around 50 users, making the timeouts/sec rate continuously grow (the orange line). At that point the machine became unresponsive and we were forced to restart it so be careful if you plan to test this in your production environment.
Then we configured and started Varnish using the default configuration file and repeated the test:
In ths case, you can observe how the initial 25ms response time decreases below the 10ms when the cache if filled and keeps stable while the users keep growing. You can also observ we did not get any error or timeout in the process.
Although these results are really impressive, as we already mentioned, to make Varnish properly handle WordPress, we should use the wordpress.vcl file. The below test repeat the process using the WordPress-specific file:
As observer, it still produces great results, with a slightly bigger response time (around 12ms in average) but it will improve how the cache is used.
