USB Raid

[Spinal]

Quick question; does anyone know if its possible to RAID a few USB hard drives together?

Our primary backup server has died, and as we have just ordered a new one (due in January) its not really getting replaced. I was thinking of ordering 4 or 5 x 500gig drives and using them as a temporary backup system. I realise it would be ludicrously slow, but it would still be faster than using tapes.

Ideas? Is it even possible?

TIA,
Michele

 

[television]

Servers are not my line ans computers are bad enough, you can get multi USB hubs, but how would you identify the one in use

 

[Spinal]

It wouldn't be running on the server... Our xServe died...

I was thinking of running this on a spare dual-G5 Mac Pro. I'm thinking of either installing OsX or some distro of Linux. Either way, I would just mount it as a network volume and have bacula (or similar) running on a remote machine.

Michele

 

[A-AvantGarde]

I think it may be possible, IIRC someone tried to make a RAID array out of 4 ipod Shuffles!

Storage is one of the things I know about in my line of work, although that's at the higher end. Why not look at an entry level NAS solution, there are quite a few drive chasis available that you then present to the network, some of them are pretty cheap now too, they're really becoming SOHO (small office / home office) type solutions.

Have a look at:

http://www.morecomputers.co.uk/extra.asp?pn=TS-H1.0TGL/R5-1&referer=bizrate

There's also a similar US Robotics device however I haven't found a UK seller for this. I don't normally recommend Netgear, but they have this (apparantly they only brand this EasyNAS system), http://www.morecomputers.co.uk/extra.asp?pn=RND4450-100EUS

There is also a Netgear one which takes just two drives (think it's called the NC-101) definitely avoid that.

Just found the USR one I mentioned:

http://www.kustompcs.co.uk/acatalog/info_3808.html

I've been looking at getting one of these three for home to centralise my storage.

 

[Dryce]

Storage is one of the things I know about in my line of work, although that's at the higher end. Why not look at an entry level NAS solution, there are quite a few drive chasis available that you then present to the network, some of them are pretty cheap now too, they're really becoming SOHO (small office / home office) type solutions.

This concurs with my view.

I just acquired a cheap NAS unit to act as an interim solution after a tape library failure this weekend.

I don't plan to go back to tapes except for specific archiving purposes. As a more permanent solution I'm looking at using a multi-drive chassis to set up a RAID as a first line backup store located in a separate building linked by fibre and either use smaller NAS units or external 0.5Gb or 1Tb USB drives for second line stores in a data safe in a third building plus additional third line units periodically transported off site.

One advantage of using NAS based units rather than USB is that the file stores are not specific to the operating system. So you're not tied to FAT or NTFS. You can access the device independently from Linux, Apple, Unix, and Windows using file shares and in some units other protocols such as FTP, HTTP, and NFS.

 

[Spinal]

Thanks for the ideas - I was leaning away from NAS mainly because (being/working for a school) we try to re-use as much as possible. I was hoping to use the USB drives for a few months and then "hand them down" to the yearbook people (for some odd reason they use MASSIVE files).

We've already ordered a RAID with 500gig drives and an xServe as our permanent backup solution; so our budget is skinny

Regarding the actual file system I'm not too fussed; we work only with macs and as such HFS is our only option really. At the moment, excluing music and video (and the students - their data isn't mission critical) I'm looking at only 500-something gigs so I probably could get away with a single 750gig drive.

I also had a deeper look at bacula - it doesn't do exactly what I need so I think I'll stick to a homebrew bash script (rysncx). I know there are issues with it and some OsX files - but for 2 months it isn't the end of the world.

I just thought of tying this into Active Directory so that users can acess the files themselves; (but on second thought, this is probably better for the long-term solution)

I just read on one of the Apple forums of someone that RAID'ed several USB memory sticks and another that did it with floppy disk drives (why!?). It seems that as long as the HD's are formatted as HFS, disc utility should be able to handle basic raid striping or mirroring. I'll have to look at that tomorrow (to lazy to go grab a laptop now)

Michele

 

[bolide]

A G5 will take a few internal drives. Bare drives are pretty cheap so I'd throw a couple of drives inside the G5 and leave it at that

SoftRAID is good for this kind of thing as it's more flexible than DU

Nick Froome
www.pvision.co.uk

 

[HughJarse]

I think you can do it with SME server. WE use SME in schools and use USB for backups - I believe it needs a patch to do it though. SME is free too.

 

[bolide]

BTW Apple certified 750 Gb drives for the Xserve RAID some time ago

Also, wrt NAS units: AFP support (for Macs) on many NAS units is appalling. You really have to have AFP 3.1. Many only support 2.0. The result with 2.0 is no long file names, truncated filenames when copying, and no support for some characters legal in filenames on the Mac OS

It would take a perfect result in torture tests to make me even consider a NAS for use with Macs. And even then you couldn't guarantee that a problem wouldn't rear its ugly head some time later - like when the OS is updated...

If you like to reuse kit then a NAS is less appealing than a G5 with a couple of HDs and a copy of SharePoints. That setup's good for <=10 users. Above that you need OSX Server if you want network-connected users

Final thing: those natty triple- and quad-interface La Cie drives (320 Gb, 500 Gb, 1 Tb, 2 Tb) are great on paper but too unreliable for server-level useage. I've stopped using them for backup and I wouldn't have live files on them under any circumstances

Nick Froome
www.pvision.co.uk

 

[Spinal]

Nick; thanks for the advice! I didn't know that the NAS drives were so problematic (having never really owned one).

OsX server will be for our 'proper' backup - I'm just trying to avoid being in the position to tell people "sorry, nothing is backed up until January".

(oh, and we did consider 750's in the raid; but the extra space wasn't enough to get us over the price difference.) We don't have THAT much data (I make it a policy not to backup movies and audio) so the sevel terabytes offered by the 500's allows me to keep a 24hr, 48hr and 7 day backup.

Michele

 

[Dryce]

It would take a perfect result in torture tests to make me even consider a NAS for use with Macs. And even then you couldn't guarantee that a problem wouldn't rear its ugly head some time later - like when the OS is updated...
http://www.pvision.co.uk

Nice thing about Macs these days is that you always have a get out clause with the Unix derived command line stuff (scp, tar, etc.).

 

[NiallerS320]

Spinal

I'm topping out at 2gb a min over gigabit using an LTO3 Tape drive (SCSI Atto UL3d card) on a G4 (server is xserve/xraid/fibre) and Retrospect (awful program), I back up a couple of tb's of data at the weekend and then alternate the days with file modification dates that have changed in the previous 24 hours. Personally the 4x500gb drives into a G5 would certainly solve your problem and be cheaper than the USB drives and much faster over sata and that should do ya till january, I'm not sure if softraid (disk utility) will allow you to raid externally mounted HDs but it'll definitely do the internal drives, my personal experience is that Hitachi & Seagate drives seem to do the job (I've lost to many Western Digital & Maxtor). If you want an external RAID enclosure you can pick em up cheap which you can take off premises which might be worth considering in case you ever need to pull in a disaster recovery plan (sox compliance ******).

Oh and bollide did you consider connecting to the NAS boxes with smb:// or cifs? and to be honest I slapped the odd filenaming conventions outta everybody no * / \ & . . in filenames, they can use _ and that's about it.

N

 

[league67]

The only usb raid I've ever used was lacie with 4 drives (r5+hs) slow as hell (connected to xserve). Atto 4ld card + infortrend RL6 anytime.

What most admins choose to ignore is that once data disappears NO ONE is going to thank you for saving money. Or get it in writing that you warned them of consequences.

As for the backups, Retrospect is not that bad, specially on recovery time. Try recovery on Legato Networker.

Back to original post. I'd use usb drive as a secondary copy. Using it as only copy is asking for trouble.

my 2p

 

[agatward]

The other option is to get an eSATA card for the G5, and an eSATA external case with some disks in it. Some cards support 4 disks on a single port (and cable), some require a cable per port.

The advantage here is it's native SATA, so no bridges and no loss of performance; and you can re-use the disks once your new xserve arrives

 

[miro]

http://www.drobo.com/products_demo.aspx

Seems like a cute product.

 

[Spinal]

http://www.drobo.com/products_demo.aspx

Seems like a cute product.

I had seen that a few weeks ago and though its VERY interesting, I'm not convinced. Timewise, I think the data "recovery" when you remove a drive will take time - and this seems verified by the movie hes watching - it seems to skip around an aweful lot...

Also, I thought drobo was USB only and didn't have a network port (not that in my case it makes a difference - but with a tool like that NAS would be nice!)

My main concern is where the redundant data is stored... if you can remove any one drive then I would see something like RAID5 - but he removes 2; so either it moves the "missing" data to the free space or it keeps alot more redundant data.

Either way, either you are "losing" space by redundant data filling up your drives or you are losing space because if you use that free space it wont have any space to use as swap space...

I would need to see it in action!

Re the backup, I'm backing up the most important data to a 500gig external (just in case) using scp (set as a cron job). I'll then see if I can get a solution sorted for the rest of the data....

Michele

 

[A-AvantGarde]

http://www.drobo.com/products_demo.aspx

Seems like a cute product.

A colleague of mine has been raving on about this. It's a shame that the interface between the device & the host is only USB and not network / ethernet.

 

[miro]

I had seen that a few weeks ago and though its VERY interesting, I'm not convinced. Timewise, I think the data "recovery" when you remove a drive will take time - and this seems verified by the movie hes watching - it seems to skip around an aweful lot...

Also, I thought drobo was USB only and didn't have a network port (not that in my case it makes a difference - but with a tool like that NAS would be nice!)

My main concern is where the redundant data is stored... if you can remove any one drive then I would see something like RAID5 - but he removes 2; so either it moves the "missing" data to the free space or it keeps alot more redundant data.

Either way, either you are "losing" space by redundant data filling up your drives or you are losing space because if you use that free space it wont have any space to use as swap space...

I would need to see it in action!

Re the backup, I'm backing up the most important data to a 500gig external (just in case) using scp (set as a cron job). I'll then see if I can get a solution sorted for the rest of the data....

Michele

This is now the king of NAS drives .......

(the yellowmachine rebadged .... the former Netgear guys who setup Yellowmachine appear to have sold out to Netgear).

http://www.netgear.com./Products/Storage.aspx


If you think that data recovery is some kind of problem then data loss sucks quite a bit more. By definition, recovery is very time consuming. The only way to recover RAID in minutes is to use fibrechannel and SAS drive on a SAN. Just add "000" to the price of a NAS.

The drobo has a nice design to reduce air flow noise. I thought that was clever.

 

[Spinal]

Slightly OT here, last nights manual backup went well, so I decided to do it via cron -problem is, I can't seem to get a private/public key set up and its driving me insane!

This what I did (on my local machine, logged in as spinal with admin rights):

ssh-keygen -t rsa 
<enter 3 times to get a default, black passkey setup>
scp ~/.ssh/id_scp.pub root@server:/Volumes/data/myTemp/
<enter>
<root@server password><enter>
<file copied successfully>
ssh root@server -l root <enter>
<root@server password>
cat /Volumes/data/myTemp/id_rsa.pub >> ~/.ssh/authorized_keys <enter>
exit <enter>
exit <enter>

Next, I tried to ssh into server several ways:
ssh server
ssh spinal@server
ssh root@server

All ask me for a password; which makes scripting it as a cron job harder... ideas?

EDIT: Sorted, I was just being stupid!
authorized_keys != authorised_keys

 

[A-AvantGarde]

This is now the king of NAS drives .......

(the yellowmachine rebadged .... the former Netgear guys who setup Yellowmachine appear to have sold out to Netgear).

http://www.netgear.com./Products/Storage.aspx

That's what I really want for my home network, or the USR one I posted (Santa please take note ). Can't justify the cost right now, I may start with a 2 drive RAID 1 solution. Currently thinking about an HP Media Vault, they seem to have good review and decent offers as I think they've discontinued the 300GB model I was looking at.