ENTRIES
Welcome to Eric Cheng's online journal! You are not logged in. [ Log in ]
«  :: index ::  »

Backing up bits

:: Monday, July 26th, 2004 @ 10:55:34 am

:: Tags:

There are more than 113,000 files in my top-level photo directory (which thankfully now resides on a RAID 5 box in the closet).

Data loss terrifies me, and so I have finally have started the arduous process of archiving everything onto DVD-R. I do own a 33GB tape drive, but at $60 or more per tape, I’d rather use 7 DVDs (less than $5) to store the same amount of information.

In the immediate future dual-layer writeable DVDs will be available in quantity and presumably will begin to fall in price, and in the longer term Blu-Ray will up the storage capacities of a single disc to 25/50GB. But at some point you have to commit to a media format and start backing up.

The problem is that the rate at which my data needs is growing is surpassing the rate at which cost-effective removeable media capacities are growing. For example, it would be great to be able to back up my life onto 20 or so discs, but currently I would probably require 100 discs or more. So instead, I tend to keep my data on multiple drives in separate machines, reserving the risk of catastrophic data loss for acts of God (and more recently, the risk of viruses brought behind the firewall by roommates). That’s not good enough, I don’t think. I need to have copies stored off-site — hence, the DVD-burning binge that I’m going through right now.

I’ve thought about rsyncing my data to a remote server, but even at an upload speed of 100 KB/sec, 500GB would take roughly 57 days to upload. I suppose I could walk into a CoLo with a server already seeded with the data, banking on the fact that incremental updates won’t take up as much bandwidth…

Also, I’m amazed at how many of my friends simply do not back their data up. As much as possible, I’ve been forcing loved-ones to use mirrored hard disks in their desktop machines, which will at least account for the possibility of drive failure.

Blah blah blah. Boring content today. But it’s what is on my mind. :)

Popularity: 2% | San Francisco, CA | link | trackback | Jul 26, 2004 10:55:34
  • syndromes

    Well, consider your job partially done – your entries have spurred me to create a dedicated backup server. I’ll just be doing raid 1, but better than what I (don’t) do now.

    I’ll also be getting a DL-DVD burner, but i’m not sure that’ll i’ll actually be dilligent in backing them up there as well. I’m lazy ya know :)

  • guest

    i am also terrified at data loss with my digital photos. i recently had a hard drive crash on me and long story short – i’m also burning my archive to dvd-r.

    i’ve noticed, on occasion, if i do a binary comparison of the images i’ve burned compared to their source, some get corrupted and i’ll have re-burn the disk. what tool do you use, if any, for binary comparisons? i’ve used winmerge, but i’ve noticed that it’s not always accurate.

  • echeng

    hey, guest. i use the “verify” checkbox in Nero. never had it fail. although interestingly, with one drive, i consistely get failures when it tries to finalize a DVD-R. works fine from my thinkpad’s multi-drive, though.

  • Alex

    I had that finalize DVD failures with my nec dvdr. I just bought a plextor that “seems” to work fine, but i still haven’t used it enough to be sure.

    I should work out some kind of mirroring for myself, at least on one of my drives (i only have ~ 100GB of important data :) but it’s growing faster now that i have a dslr

  • vinh

    :) “Data loss terrifies me” from the same guy that disabled the diskhead parking safety feature on his laptop.

    the DVDs are just like seeding the a remote server over a low-bandwidth line, once you’ve gotten past the initial set of burning, so long as you continually burn data incrementally, it won’t be so bad. it’s only bad now because you hadn’t taken any backups until now, so this one is a full backup.

    take the time to catalog these DVDs now while you’re burning… otherwise, once you’ve amassed a library it’s gonna be a PITA to find anything.

  • echeng

    do you have a suggestion for a file cataloging program? i’m not interested in cataloging my local volumes. i only want to catalog the DVDs i’m burning…

  • vinh

    =\ unfortunately no… i’ve been meaning to either find or come up with something myself for a long time now… i archive so much stuff onto CDs, but can never remember what CD it’s on…

    in your case, whatever program you find or script out yourself, assuming that you’re already using some kind of asset library/manager, you should look for something that can pull the descriptions or captions you have for your images… the idea being that you’d want something where you can search for a term and pull up a list of files and what CD/DVDs they’re on.

    and if you’re too lazy to look for something, keep an excel spreadsheet or access database with a CD/DVD name and a description of its contents. :)

  • stephane

    FWIW you can see the equipment that Bjørn Rørslett is using. Gee, those pros are scary.

  • stephane

    looks like b2 does not support utf-8 so read that encoded data as ‘Bjorn Rorslett’

blog comments powered by Disqus
RECENT TWITTER ACTIVITY
ARCHIVES
Journal Home
Where is Eric? (password)
Stuff for Sale
February 2012 (3)
January 2012 (6)
December 2011 (8)
November 2011 (10)
October 2011 (12)
September 2011 (8)
August 2011 (14)
July 2011 (9)
June 2011 (9)
May 2011 (11)
April 2011 (11)
March 2011 (12)
February 2011 (23)
January 2011 (22)
December 2010 (16)
November 2010 (17)
October 2010 (26)
September 2010 (24)
August 2010 (24)
July 2010 (30)
June 2010 (26)
May 2010 (21)
April 2010 (26)
March 2010 (19)
February 2010 (17)
January 2010 (29)
December 2009 (21)
November 2009 (23)
October 2009 (32)
September 2009 (19)
August 2009 (34)
July 2009 (21)
June 2009 (30)
May 2009 (23)
April 2009 (18)
March 2009 (6)
February 2009 (25)
January 2009 (5)
December 2008 (6)
November 2008 (22)
October 2008 (27)
September 2008 (25)
August 2008 (34)
July 2008 (34)
June 2008 (32)
May 2008 (26)
April 2008 (15)
March 2008 (19)
February 2008 (31)
January 2008 (43)
December 2007 (33)
November 2007 (29)
October 2007 (29)
September 2007 (9)
August 2007 (19)
July 2007 (10)
June 2007 (17)
May 2007 (26)
April 2007 (38)
March 2007 (39)
February 2007 (13)
January 2007 (35)
December 2006 (35)
November 2006 (14)
October 2006 (6)
September 2006 (20)
August 2006 (24)
July 2006 (32)
June 2006 (17)
May 2006 (23)
April 2006 (16)
March 2006 (16)
February 2006 (26)
January 2006 (34)
December 2005 (17)
November 2005 (21)
October 2005 (18)
September 2005 (17)
August 2005 (5)
July 2005 (15)
June 2005 (20)
May 2005 (25)
April 2005 (7)
March 2005 (22)
February 2005 (20)
January 2005 (38)
December 2004 (6)
November 2004 (24)
October 2004 (16)
September 2004 (22)
August 2004 (12)
July 2004 (17)
June 2004 (15)
May 2004 (11)
April 2004 (35)
March 2004 (40)
February 2004 (29)
January 2004 (36)
December 2003 (20)
November 2003 (18)
October 2003 (10)
September 2003 (18)
August 2003 (10)
July 2003 (34)
June 2003 (12)
May 2003 (49)
April 2003 (42)
March 2003 (42)
February 2003 (15)
January 2003 (7)
December 2002 (17)
November 2002 (19)
October 2002 (24)
September 2002 (22)
August 2002 (20)
July 2002 (21)
June 2002 (14)
May 2002 (15)
April 2002 (11)
March 2002 (13)
February 2002 (20)
January 2002 (17)
December 2001 (16)
Even Older Journal
Travel Journals

CATEGORIES / TAGS
(25) (2) (2) (1) (1) (6) (2) (2) (11) (7) (3) (1) (1) (2) (4) (1) (6) (1) (1) (1) (6) (2) (1) (3) (1) (5) (1) (20) (1) (1) (1) (1) (14) (10) (1) (1) (1) (1) (27) (6) (3) (2) (4) (1) (41) (11) (9) (4) (38) (1) (3) (2) (4) (1) (1) (1) (2) (1) (1) (9) (22) (8) (3) (2) (3) (2) (1) (3) (1) (1) (1) (4) (1) (1) (5) (43) (1) (3) (22) (1) (1) (1) (1) (1) (10) (3) (1) (1) (1) (1) (6) (5) (1) (1) (1) (1) (3) (1) (1) (69) (3) (3) (7) (1) (1) (15) (6) (1) (28) (1) (7) (1) (3) (4) (4) (1) (1) (1) (1) (1) (8) (4) (3) (2) (89) (13) (2) (77) (2) (2) (1) (1) (1) (1) (1) (3) (2) (3) (1) (1) (24) (3) (5) (2) (1) (2)


Eric Cheng's RSS Journal Journal RSS
Eric Cheng's RSS Journal Comments RSS

proudly powered by wordpress
script exec time: 0.86s
i hate computers.