The last couple of weeks I’ve been getting accustomed to Google Fusion Tables.

For those who haven’t yet noticed it – Fusion Tables is a new (still in Beta) Google product for data storage: a kind of database.

Some major cool things about Fusion Tables:

1. it’s really accessible (use the usual OAuth2 authentication as for all other Google services to script interaction), or use the web interface to manually edit your tables.
2. If you include geo data in your tables, you can easily (and I mean really easily!) display your data on a map.
3. They’re really clever about geo data… a text address, longitude/latitude, polygons, shapes, … dump it into a “Location” type field, and it will probably be understood.

Net::OAuth2 isn’t actually a new Perl module, but in its previous state it wasn’t really usable for command line applications, so I reworked it a bit.  Both of these modules are very raw/alpha – they are lacking in documentation, error handling, unit tests…. all the things one would expect in productive code… but I want to get them out there now so other people can start using and abusing them.

Here’s a wee screencast of pfusion in action:

And a short instruction to install (I hope it’s complete…):

git clone git://github.com/robin13/Net-OAuth2.git
git clone git://github.com/robin13/Google-Fusion.git
vi ~/.fusion

Edit to look like this, with the id/secret of your application.  If you haven’t got an id/secret yet, you’ll have to register an application for this purpose on the Google API Console and be sure to make it an “installed application” (not “web application”)

---
client_id: 372296649547.apps.googleusercontent.com
client_secret: XDy8Y8xTuwN90F3h7ljCuw4i

# And now run fusion:

perl -I ./Google-Fusion/lib -I ./Net-OAuth2/lib ./Google-Fusion/bin/pfusion

If authentication is successful, you should be able to interact with Fusion Tables to your hearts content, and should find a .fusion.auth file in your home directory with your current tokens (you only have to enter the authentication code the first time).

I hope you have fun with it, and give me some nice pull requests on those repositories!

A typical problem with a home server/NAS used by several people is that someone always turns it on, but who turns it off?  It stays on for days, and the electricity meter whirs happily as the kilowatts get unnecessarily burned…

A better strategy is to let the server shut itself down when it is not being used… intelligently… and this is how I made it happen:

How to determine if the server is not being used

This depends a lot on your setup.  In my case I have a Linux NAS with 6 disks.  One disk (/dev/sda) is the operating system (Linux) disk, and the other 5 are in a (software) raid array serving the shared filesystem which I and my flatmates and I use.  This makes it pretty easy to see if someone is using the system because if none of the 5 raid disks are active, then no one can be using them.  You can see if they are active with hdparm.

A brief introduction to hdparm

hdparm is a really handy linux utility for setting and reading hard disk parameters/status.  To set the spindown time of a disk to 20 minutes:

hdparm -S 240 /dev/sdb

To make your settings persistent, put them into /etc/hdparm.conf

/dev/sdb {
spindown_time = 240
}

To see the current state (active or not) of a disk:

hdparm -C /dev/sdb

Look up the hdparm documentation for all options.

How to shut down the server…

For this we need a daemon which monitors the hard drive status, and initiates a shutdown when the desired… Introducing: Shutdown Daemon (SDD).

This is a script I put together to do exactly what I described above.  For my setup I have used the following configuration:

---
log_level: INFO
log_file: /var/log/sdd.log
sleep_before_run: 60
monitor:
 hdparm:
 loop_sleep: 60
 trigger_time: 3600
 disks:
  - /dev/sdb
  - /dev/sdc
  - /dev/sdd
  - /dev/sde
  - /dev/sdf

This means it waits one minute before starting monitoring, checks the disk status every minute, and will wait for an hour of spun-down disk status before actually shutting down the server (unless one of the disks becomes active again within that time).

Installing SDD

Clone sdd from the github repository:

git clone git@github.com:robin13/sdd.git

Make it:

cd sdd
perl Makefile.PL
make
make test
make install

Copy the init.d script and config, and update the rc.d entries:

cp examples/etc/sdd.conf /etc/sdd.conf
vim /etc/sdd.conf
cp examples/init.d/sdd /etc/init.d/sdd
update-rc.d sdd defaults
/etc/init.d/sdd start

Saving power

In my case, just configuring the spindown on the hard drives reduces the power consumption of the NAS server from 44 to 28Watt.  Now factor in all those times where the server was forgotten and left on for days… the power saving potential is fantastic!

What other monitors could be useful?

Of course spindown time of the disks isn’t the only condition which could be useful for shutting down your server.  Here’s some others I’ve thought of:

• hard drive temperature too high
• No user logged in with ssh (users)
• No locks on samba file shares (smbstatus -L)
• …

If you have any ideas, feel free to fork the project on github and implement a Monitor – I’m looking forward to seeing many more monitors! 🙂

Varnish is the cutting edge in reverse proxy for speeding up web services and reducing load on your web application server.  It has a pretty neat CLI for managing a running Varnish instance, but being an interactive telnet interface, it is not so easy to script for…

Some examples why you might want to interface to the Varnish CLI with a script:

• Automatic reload of configuration file when it changes
• Purge the cache for a specific website when changes are made (new blog entry, new CSS, …)
• …

And so I wrote up a Perl interface to the Varnish CLI (git repo here).  The module is Moosey, and requires Net::Telnet and Digest::SHA to run.

It’s pretty easy to use – here’s a snippet with which you can purge your cache:

use Varnish::CLI;
my $v = Varnish::CLI->new();
$v->send( "url.purge .*" );
$v->close();

Have fun, and keep your web services zipping with Varnish! 🙂