Robin Clarke – Life and Tech

A typical problem which many web service providers have is that when they transfer a website from one hosting to another, there is an inevitable period of time when the application has been transferred, but the DNS changes have not propagated, and so some users will experience the site as being offline.  You can reduce the effects by reducing the TTL for your domain prior to the switch, but I prefer this method.

The key to this method is Varnish.  Varnish is a state of the art reverse proxy, and you should be using it to accelerate your web applications, even if you don’t need to switch service provider.  Explaining Varnish in detail goes beyond the scope of this howto, but in summary, clients connect to the Varnish proxy on the server, which (based on a vcl configuration file) either servers from its cache, or forwards the requests to backend servers.  This can allow you to have a farm of web servers all with one frontend, or just take the load of your backend servers by efficiently serving static content .  Varnish also allows complex filtering, and this is what makes the transition from one service provider to the next a piece of cake.

For the following I’ll use the terms “old” as your old ISP server, and “new” for your new ISP server.

Here’s the basic steps:

• Set up Varnish in front of your web server redirecting all requests to the local backend.
• Add a new backend in your Varnish configuration, defining your new IP as the host.
• For each application (typically a domain):
  • Copy files and databases from old to new
  • Install the files and databases on new.
  • Test the new server (make a static entry for the domain(s) in your hosts file)
    • Are all required modules installed?
    • Is everything working as expected?
  • Add a rule in your Varnish configuration that will forward all requests to this domain to your new server.
  • If you have a very active site, you may still have a few seconds of downtime in this step…
    • Put the web application into maintenance mode
    • Dump the database from old again (to get latest changes since your testing)
    • Load the database into new
    • Reload the varnish configuration to activate the forwarding to the new server
    • On the new server, take the application out of maintenance mode
  • Now all requests which hit your old server for this domain should be invisibly forwarded to your new ISP
  • Check that everything is working
  • Change the DNS entries for the domain – as this propagates, less traffic should be hitting your old ISP
• When you see no more traffic hitting your old server, you can turn it off.

Using this procedure, you can comfortably move any number of applications with minimal downtime, and enough time to test each one as you go.  It also will allow you to transfer applications where you do not have control of the DNS entries: you can do the transfer, and then inform the domain owner to change the ANAME entrie for their domains.

Some situations where this might cause problems:

• Your application has IP based filters or bot-blocking active – if so your new server will be getting a lot of traffic from one address (old server)
• Your application is not compatible with Varnish?  Some web applications (Squirrelmail for example) generate many unnecessary or duplicate HTTP headers.  Varnish chomps everything after the first 32 lines of HTTP headers, which in turn can cause the application to fail if something important was after the 32nd header…

You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.