e-Things

There are occasional times when being known as the “Twitter fan” in the Web team here, can turn out something of a mixed blessing… Read the rest of this entry »

In my previous post, I started looking at the question of backing up your data: why it’s a good idea to do this regularly (though many of us don’t until it’s too late), and a couple of methods one could use to do it.

I was thinking last time in terms of “local” backup (i.e. to a drive or other medium connected directly to your computer), but it’s also generally a good idea (at least in principle) to protect your most valuable data by keeping a copy off-site, in case of fire, flood or similar events. (Incidentally: as with my last post, I should stress that your IT department should have its own data backup arrangements in place, so most of what you will read here would apply mainly to your own personal use. On the other hand, I emphasised the word “should” deliberately there—as I have found in the not-so-distant past, it does no harm to make sure that you really are “covered”.)

One simple and fairly obvious off-site backup method, would be to arrange to leave your storage medium (CD, DVD or other drive) with a friend or colleague, although you would then have to trust them with the data, or find a way to encrypt or otherwise protect it.

Another option is the increasing number of online storage services available, such as Dropbox, BT’s FileVault and Apple’s iDisk. These offer varying levels of convenience and security, and the advantage that you can access them from any computer with an internet connection. However, most of these services charge for storage space above a few gigabytes in size, so you may not be able to store more than your most important files there.

Furthermore, file transfers will only be as fast as your internet connection, and if you are at home, your ISP may have a data usage (bandwidth) limit on your connection, so be careful when transferring large amounts of data (an incremental backup program is useful here). You should also check whether a service supports your computing platform, particularly if you are a Mac or Linux PC user—some online backup services are either Windows-only, or have limited support for other platforms.

One issue you should consider—particularly, but not exclusively, if your organisation falls under the Data Protection Act, and/or if the data is at all personal—is protection of your backup data by encrypting it. This is especially important if any data is to be held off-site in any form, and depending on your organisation’s IT policy, it may even be compulsory to encrypt your backups in case they may be accessed by unauthorised parties.

There are many encryption applications out there, but if you would like to try one yourself, I recommend TrueCrypt as a first point of call. It is an open-source program, free to use and available for Windows XP (and later), Mac OS X and Linux, and works by encrypting entire drives or partitions rather than individual files. Once you understand the concepts, TrueCrypt is a useful way to protect your backup data, using strong encryption methods which cannot be easily broken.

The area of backups is one where we could go into as much or as little depth as we wish, but the principle remains: Make sure your important data is being backed up, and regularly. Of course, though, unless you actually do something, it will stay a principle, at least until your hitherto reliable hard drive goes up in smoke…

I suspect that at this very moment, I could divide those reading this post into one of two categories:

1. those who are wondering why I’m writing a post about the importance of backing up data; and
2. those who weren’t sufficiently careful about it in the past, and having lost important data in the process, have a pretty good idea why I’m writing a post about the importance of backing up data.

The American IT podcaster Bill Blinn (“TechByter”), wrote in an online conversation that “any data that is in just one location, doesn’t exist“. Having recently lost some important material due to an incomplete backup, whilst the statement is not literally true, I understand where the commentator was coming from.

Whilst ensuring your data has at least one up-to-date copy at all times, is perhaps not the most exciting chore in the world (let’s be honest), that doesn’t mean it’s any less vital for your computing activities, and everyone should have some kind of backup routine (and not just an ad-hoc, “when I feel like it” activity) in place in case the worst should happen.

Now, you may well be fortunate enough to have an IT administrator or team who is taking care of this for you, but what if you don’t? Well, you may wish to start by looking at the data you hold, and imagine what you would miss most if the media it was held on, suddenly went up in smoke (figuratively or literally). This isn’t as unlikely as you might think: hard drives are more reliable than ever these days, but they can still fail on you, and (it seems) often at the worst possible time.

Then, consider how you might ensure you have a second copy of this data, and for that matter, a copy which is as close as possible to the present time. A six-month-old copy is better than nothing, but surely you’d prefer one which was less than a week old?

There are all sorts of ways in which you could take backups (and keep on taking them), but here are just two relatively simple ones:

• Copy the file(s) and/or folder(s) onto another storage device (portable hard drive, USB memory stick, recordable/rewritable CD/DVD, etc.).
• Use an incremental backup program, which will backup only the changes made and add them to data it already holds. If your computer allows it, schedule the backup to take place automatically on a regular basis. (There are plenty of incremental backup apps for Windows; Linux/UNIX machines have “rsync”; and if you have a Apple Mac running Mac OS X 10.5 or later, the “Time Machine” feature is a “must-use”. Find yourself an external USB or FireWire hard drive, and Time Machine will backup your Mac incrementally.)

One other piece of advice at this point: when you make backups, please do make sure you are backing up all the data you would need if you had to restore it. There is little worse than coming to restore from a backup, only to find that an essential directory is missing—I’m afraid this is the voice of experience speaking here…

That takes care of “local” backups, but you may also wish to consider some kind of “off-site” backup option. I will take a look at some of these in my next post, but in the meantime, I will leave you with a final thought from the aforementioned “TechByter”:

Files exist on fragile magnetic or optical media. The more copies you have, the less likely it is that you’ll lose something important.

Recently, I wrote a brief introduction to gathering and analysing information using visual tools, and promised a follow-up with some free-to-use tools you can use to delve into your data. Without further ado, let’s begin exploring…

One fine example of creative thinking with data visualisation, is IBM’s research project Many Eyes. The premise is simple: feed a data source into the tool, choose the way you’d like to view it, and you will receive your data in visual form. Many Eyes offers a pretty wide range of visualisation tools, including a number that I outlined last time (from more familiar pie, line and bar graphs, to treemaps, bubble charts and block histograms), and within those types you have quite a wide range of options for customising the output you receive.

One of the visualisation types provided by Many Eyes, is an implementation of a separate tool, Wordle, which like Many Eyes emerged from IBM research. Wordle accepts a text source as input (this can be either a file or an RSS feed), and returns a “word cloud”—a pictorial bundle of words where the size of each word reflects the number of times it appears in the text. For examples of what these look like, the Wordle Gallery page shows the most recent wordles created by users, though you may have to look through a few pages to find the most representative ones.

Wordle also offers a healthy number of options for configuring its output, including colours, typefaces and layout, and as with Many Eyes, you can send the results to any printer available to your computer. If you have a PDF “printer” application installed (such as CutePDF Writer for Windows, or Mac OS X “Leopard”’s built-in PDF option), this allows you to bring the Wordle PDF output to a professional printer, or import it into a graphics application for further work.

Both Wordle and Many Eyes require your browser to support Java, and a reasonably fast computer and/or Internet connection would be useful for best results.

Another online resource which may not have occurred to you as a tool for looking at data—in this case, for positioning—is Google Maps. One of its most powerful yet less heralded features, is to accept an RSS feed as input. The possibilities of this really start opening up when the feed in question includes geopositioning data (e.g. from a GPS unit). For example, online photo-sharing sites like Flickr and Picasa can display both the photo and a map showing where it was taken (if the photo came from a cameraphone with a GPS positioning feature).

These particular sites also export photo and positioning data for groups of images as an RSS feed, which can then be loaded into Google Maps. This way, you can view a map overlaid with the locations where a set of photos were taken. It is fairly easy to find examples on the Web of how to do this; try “flickr google maps mashup” in your search engine of choice.

Whilst on the subject, Google Earth can import and display KML files, which contain data designed for displaying on a map. Applications such as Nokia’s Sports Tracker can output data in KML format, meaning you can view the route of a journey in Google Earth (and/or Google Maps, which now also supports KML).

Google Earth has also been used to create choropleths (the geographic term for “heat maps”, which colour-code areas of a country or continent by the strength of data in that area), though professional geographers prefer dedicated GIS applications, which lie somewhat outside the scope of this article.

All these applications are available at no cost, so please feel free to try them out and see how they may be able to help you. A final word of caution, though: Wordle can become oddly addictive once you get started… or is that just me?

External links

• 16 Awesome Data Visualization (sic) Tools: it’s a big page, but points to a good number of tools I hadn’t been aware of (I think I’ll check out Fidg’t, for one).
• FUMSI – Data Visualisation Tools and Examples: Another useful article (thanks Cormac for pointing this one out), containing a healthy number of links to online data visualisation tools, many of which were new to me.