Cluster and distributed filesystems

This is a surprisingly large ecosystem for an area of OS’s that would on the face of it seem very specialised. I think there are a few  facts that are bringing these features to the mainstream:

1. we’re looking at new IT innovations like cloud computing

2. management interfaces to provide these features are being developed and simplified

3. features have stabilised

Distributed filesystems include GlusterFS, Lustre, GFS2 and Hadoop while clustered filesystems include OCFS2, GFS2 and DRBD. Red Hat Enterprise includes GFS2 allowing clustered storage solutions through simple gui tools and DRBD is now available in the canned Openfiler storage solution ( not simple to setup but there is a good tut at howtoforge ).

GlusterFS uses a very interesting technique of aggregating storage bricks ( using items like iSCSI, DAS, AoE and SAN-supplied LUNs ) in mirrored or other RAID configurations providing redundancy for data across your storage environment. Setup does not involve kernel modules or changes so it’s fairly straightforward.

ZFS is also included in the latest FreeNAS rc snapshot meaning you can slice and dice your storage exactly how you want it with writable snapshots, storage pooling and simple administration.

The upside of all these storage applications means you can provide non-stop storage across an enterprise at a very low cost and simultaneously aggregate performance for those high-bandwidth apps. Virtualisation of storage, previously the domain of high-end commercial companies like Netapp, StorAge and Falconstor, is now available for little to no cost through a whole host of FOSS and commercial OSS software.