I recently had the opportunity to review web monitoring services. I was looking specifically for a service which would test a web application, as if a real user was testing it: including DHTML/AJAX, in a real browser. There are a number of services such as serverguard24, and uptrends which do allow transactions (a complex sequence of requests), but they don’t run their macros in a real browser, and so aren’t enough like a real user. Then I found alertfox!
Alertfox
To use AlertFox, you first have to install the iMacros plugin for your browser (available for Firefox and IE). This makes recording a macro (sequence of clicks and data entries) a piece of cake. You can choose to have clicks recorded by the id of the element you are selecting, or by the X/Y coordinates. Once you have recorded a transaction (you may need to edit the script to replace session ids with wildcards (*)), you can create a “sensor” with it on the AlertFox site. A Sensor is a rule set including the macro which should be carried out, how often it should be executed, in which browser, and what the acceptable time is that this transaction should take. You can even define from which location the test should be carried out (“Europe Zone” and “World Zone” connections both currently come from servers hosted by serverloft in Germany, “US Zone” connections come from theplanet servers in Houston Texas).
When things go wrong: If the transaction cannot be completed (server is too slow, a page in the sequence is not as expected, …), you immediately get an email warning with a link to a screenshot. This is great! Setting up a host of these sensors I feel like I’ve got a horde of tireless monkeys working day and night clicking, typing, and making screenshots. On your AlertFox dashboard you have a good overview of all your sensors, when they were tested, and what the results were each time.
One small annoyance: you often won’t be able to use a macro recorded in Firefox for IE – there are small differences in how they recognise tags which sometimes break cross compatibility. Solution: record the same transaction twice – once in Firefox, and again in IE.
Why outsource monitoring?
You might wonder what the point is of outsourcing your web application monitoring: surely you can monitor server load in-house. This will cover many causes of downtime, but what if your server is running fine, but network interruption has made it unavailable from outside?
So you do need to have monitoring from a remote server… Maybe you have another server and can make both of your servers monitor each other, or maybe you want to use a basic monitoring service which makes simple http requests to test if your server is online. This is better – you will know if your server goes offline, but that doesn’t cover some of the most common problems which could occur…
One of the typical errors on a web application which won’t be caught by any of the above methods is your payment system breaking. A software error, or a glich in the payment fulfillment provider… it will allow users to visit your site, put items in the shopping cart, but not complete the checkout. Your burning marketing money bringing them to your site, and no cash is coming in. The only way to be sure your site is working is to test real transactions in a real browser, but … monkeys aren’t that clever, and employing slave labour will probably get you in trouble.
If you are an online marketer it is quite likely that you are responsible for driving traffic to a site, monitoring and optimising your campaigns without having any control over the site you are working for. If the site goes down or a section is broken you want to know asap so that you can inform your customer and stop burning marketing budget on a broken site.
What does it cost?
They have 4 packages ranging between free (only one Macro, and that only on Firefox) and $199. I used the PRO2 version for $99/month which is the cheapest version which allows testing in IE and Firefox.
Links
I spent all my time talking about AlertFox because it’s the only one which fulfils my requirements, but there are many other monitoring services which may be up to your needs if you don’t need application level monitoring (DNS, HTTP, FTP, …), and here’s a list of some of the others I looked at: