Thread: Archiving the massive raw media

Page 1 of 5 123 ... LastLast
Results 1 to 10 of 49
  1. #1 Archiving the massive raw media 
    Senior Member
    Join Date
    Jul 2012
    Posts
    827
    It is important to address the long term archiving issue now before getting overwhelmed with the massive raw data that is generated by just one feature movie.
    One archiving system came to mind is the Open LTO tape, this is an open standard that allows you to write, read and copy as you do with a disk drive, your computer looks at it as another hard drive without proprietary software.
    Let us start kicking this issue around, I am sure that we have some experts here on this forum.
    Two articles are good to read:

    A File System for Linear Digital Tape
    http://www.tvtechnology.com/prntarticle.aspx?articleid=214811

    Preserving Creative America
    http://blogs.loc.gov/digitalpreservation/2011/12/preserving-creative-america-the-academy-of-motion-picture-arts-and-sciences/
    Reply With Quote  
     

  2. #2  
    Senior Member
    Join Date
    Apr 2012
    Location
    Minneapolis, MN
    Posts
    2,885
    The big rumor is we might start seeing the first 10 layer Blu-ray recorders by next sping. This will be 250 gigs per disk. This would be the ideal archive system. It's not out yet tho, but with more cameras going RAW you'll see the demand for such a device jump.
    Reply With Quote  
     

  3. #3  
    Junior Member
    Join Date
    Aug 2012
    Posts
    4
    Making backup on 2 x hdd is faster and more efficient

    Optical drives and tapes are too slow for todays turnarounds.

    just my 2 cents.
    Reply With Quote  
     

  4. #4  
    Senior Member laco's Avatar
    Join Date
    Apr 2012
    Posts
    474
    Quote Originally Posted by Margus Voll View Post
    Making backup on 2 x hdd is faster and more efficient

    Optical drives and tapes are too slow for todays turnarounds.

    just my 2 cents.
    +1
    I'm planning backing up on multiple external hard drives.
    (btw, you're the first familiar name from the creativecow davinci forum, welcome!)
    Reply With Quote  
     

  5. #5  
    Senior Member
    Join Date
    Apr 2012
    Location
    NYC
    Posts
    411
    Quote Originally Posted by Margus Voll View Post
    Making backup on 2 x hdd is faster and more efficient

    Optical drives and tapes are too slow for todays turnarounds.

    just my 2 cents.
    Backups and archiving are two different things, though they are obviously related.

    Hard drives are good for simple backup and data redundancy, but not good for archival purposes. They are not shelf-stable, and given the way CDs and DVDs degrade, I'm pretty wary of the longevity of optical formats (though I've heard Blu-ray is more stable).

    Tape is a better option, but as with any digital archival technology, data migration after a certain period of time (usually 7-10 years) is key.

    Even a 250 GB Blu-ray is only going to carry half an hour of RAW footage from this camera, which will make archiving any significant project quite tedious.

    In terms of longevity and easy readability (both incredibly important for archival purposes), the best archival format for motion pictures is still black and white separation masters made on film, hands down. Prohibitively expensive for any independent production, though.
    Reply With Quote  
     

  6. #6  
    Senior Member
    Join Date
    Jul 2012
    Posts
    827
    Quote Originally Posted by Margus Voll View Post
    Making backup on 2 x hdd is faster and more efficient

    Optical drives and tapes are too slow for todays turnarounds.

    just my 2 cents.
    We are talking about long term archiving for asset protection, and yes you still need cashing drives ( online, nearline storage ), $ 40.00 for 1.5TB/3TB tape.
    Reply With Quote  
     

  7. #7  
    Senior Member Peter J. DeCrescenzo's Avatar
    Join Date
    Apr 2012
    Location
    Portland, Oregon USA
    Posts
    2,000
    As Margus mentions above, backing up to multiple hard drives is compelling for reasons of speed and cost. Although it's true that a hard drive sitting on a shelf for several years has a certain chance of becoming unreadable, given their low cost and speed, imagine making backups to 3, 4 or more hard drives instead of only 2.

    You can even use drives made by different manufacturers, format the drives using different computer OSs, and should of course store them in different locations (different buildings) to spread risk. Given all this redundancy, the chance of none of these backups being usable after several years might be fairly small.

    Compare the above to the much, much slower and more expensive process of making backups on tape-based systems. Remember you must make at least 2 copies of the tapes and store them in separate buildings, too, just like hard drives. And later, accessing data on tape systems is very slow & expensive compared to hard drives.

    I agree and totally understand that for medium-to-big budget productions backing up to tape probably makes sense. But for most productions, "deep" hard drive based backups (3-6 copies) will probably be less expensive, faster-to-create, faster-to-access, and possibly as reliable (better?) as tape based systems.

    Also, the "real" value of a production sometimes becomes apparent a few years after it's shot. At that point one could transfer the hard drive backup to a tape or other system if money materializes to pay for it.
    Last edited by Peter J. DeCrescenzo; 08-08-2012 at 04:11 PM.
    Reply With Quote  
     

  8. #8  
    Senior Member
    Join Date
    Jul 2012
    Posts
    827
    " Going tapeless ? not so fast ... "

    http://www.nxtbook.com/nxtbooks/newbay/tvt_ltfs/index.php#/2
    Reply With Quote  
     

  9. #9  
    Member tefliux's Avatar
    Join Date
    May 2012
    Location
    Luxembourg
    Posts
    74
    A good solution would be to buy a Tandberg LTO-5 HH Fiber Channel External drive and the Promise SANLink thunderbolt to FB if you are a mac user like me.

    It costs about 3000 € but a tape cost is only 60 € for 1.5 TB. And this is really an ARCHIVING system.
    Reply With Quote  
     

  10. #10  
    Senior Member Frank Glencairn's Avatar
    Join Date
    Apr 2012
    Location
    Germany
    Posts
    3,774
    Quote Originally Posted by Peter J. DeCrescenzo View Post
    Also, the "real" value of a production sometimes becomes apparent a few years after it's shot. At that point one could transfer the hard drive backup to a tape or other system if money materializes to pay for it.
    Yeah, in both directions.
    Some of the "precious" material we shot in 3:4 and even 16:9 SD 10 years ago, is still in good shape on those Betacam tapes - but who cares? Most TV stations would not even accept it today.
    If you don't have some super interesting documentary or something important, it's not worth a dime.
    Especially advertising, corporate and industrial material has a half live of less than a year.
    Some ENG stuff and newsreels may be interesting for future generations, but even most music videos are not exactly worth it, to think about long term storage.



    The amount of media we all generate every day is bigger than ever, and so is the part of worthless crap -just think what get's uploaded to Youtube every day.

    Actually archiving digital doesn't really work anyway, In 20 years you not gonna find a drive, a connection, a computer and a program that can read your ancient codec.
    If you want REAL long term archiving, you have to print it on film (3 x b/w for R-G-B) or transcode it to the latest codec and new drives/tapes every few years.

    Edit: and yeah, clay tablets, buried under sand , has proofed to be the best long term storage - a bit cumbersome for film though
    Last edited by Frank Glencairn; 08-08-2012 at 05:14 PM.
    Reply With Quote  
     

Bookmarks
Bookmarks
Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •