Thread: Fastest Way To Offload Footage

Page 1 of 3 123 LastLast
Results 1 to 10 of 29
  1. #1 Fastest Way To Offload Footage 
    Junior Member
    Join Date
    Aug 2012
    Posts
    21
    I am need of figuring out a faster way to offload footage from my 256GB cards in the field. What would be a good, fast and portable way to do that in all of your experience? What is a good fast reader?
    Reply With Quote  
     

  2. #2  
    Junior Member
    Join Date
    Mar 2017
    Location
    Sydney, Australia
    Posts
    14
    I use the Lexar Professional Thunderbolt hub because I can download multiple cards at once without slowing down, and I use Hedge (http://hedgeformac.com) to download my cards. The only problem with the Lexar hub is that it requires power, but each card reader can be removed and plugged straight into the computer via USB3 if you're battery only.
    Reply With Quote  
     

  3. #3  
    Senior Member DPStewart's Avatar
    Join Date
    Feb 2015
    Location
    The Desert
    Posts
    3,101
    Well...there's FAST, and there's also RELIABLE.

    (And there's also CHEAP!)

    I picked up a small $275 ACER laptop from B&H. ..small as in an 11-inch screen only. Very lightweight too.
    I put a 1T SSD in it and all the basic apps leave me about 800Gig for daily transfers.

    I like this because I can usa a data-verification app (TeraCopy) for the transfer, and then also immediately VIEW the files too.

    I guess this is not the fastest nor the highest capacity, but it's convenient, useful, EASY, and reliable.

    ... for when you've got to be SURE before leaving the set. (Which is ALWAYS.)
    Cameras: Blackmagic Cinema Camera, Blackmagic Pocket Camera (x2), Panasonic GH2 (x2), Sony RX100 ii, Canon 6D, Canon T2i,
    Mics: Sennheiser, AKG, Shure, Sanken, Audio-Technica, Audix
    Lights: Every Chinese clone you can imagine
    Reply With Quote  
     

  4. #4  
    assuming youre using c fast cards this is what i use. it will be expensive

    https://www.bhphotovideo.com/c/produ...ssd_usb_c.html

    http://www.ebay.com/itm/CFast-Card-R...8AAOSw3KFWeR~9
    Reply With Quote  
     

  5. #5  
    Well, the best thing to do is identify the bottlenecks in your workflow. Ideally you'd have a:

    - Computer with several high speed, independently controlled busses; namely, Thunderbolt 1/2/3 or USB 3.1 Gen1/2.
    - Media reader that can connect to one of those high speed busses.
    - Media management software that can offload to 2+ locations at the same time (Pomfort Silverstack, ShotPut Pro, etc.).
    - At least two destination drives with a high read/write speed.

    That last one trips most people up, because independent content producers are always working on tight budgets. Sure, that portable Seagate 4TB HDD might seem like a great buy at first but when you run into its ~100MB/s throughput you'll start to realize why most good data managers build carts with RAIDs, SSDs, or often both simultaneously! Here's the advice that I give most people when they're managing data:

    1. All hard drives--regardless of the technology in them--will fail, and so it is totally unacceptable and reckless to allow data to exist in only one place. Immediately back up your footage in two places before you lose it. Preferably, find ways to make sure that copies of your data are also stored in different geographic locations to minimize the impact of natural disasters, house fires, etc.

    2. Never fill a drive of any kind past 90% full. Filling drives typically means slower throughput, and it also limits your drives' ability to write around bad sectors/flash memory.

    3. Imagine you say a phrase to one of your friends (e.g. "The quick brown fox jumps over the lazy dog.") and they have to repeat it to someone else from memory. That friend then repeats it to another friend, who repeats it to another friend, who repeats it to another friend. By Friend #24, there's a good chance that the words will have been mixed up to create a different phrase (e.g. "The speedy red cat leapt over the sleepy pup."). Now, imagine instead that when you initially said the phrase to your friend, you also gave them a piece of paper that had the phrase written on it. Along with this phrase, the paper would get passed along the chain too so that each person who heard the phrase could compare it to what was written so that they could be sure what they were being told was accurate. That piece of paper is performing the same basic function that a checksum does. Whenever you offload, always perform a checksum so that you can be sure the data you're storing is the same as the data that came right off the camera. Try to rely on XXHash-64 or CRC32 checksums to ensure the highest throughput during the transfer process.

    4. The three important checks you need to make during offloads are filesize (make sure everything was copied!), checksum (make sure everything was copied right), and visual verification (make sure the camera got what you needed in the first place). Each type of examination is a critical part of the offload process and shouldn't be neglected!
    Last edited by Alex.Mitchell; 03-20-2017 at 12:45 AM.
    Reply With Quote  
     

  6. #6  
    Senior Member rick.lang's Avatar
    Join Date
    Apr 2012
    Location
    Victoria BC Canada
    Posts
    3,956
    Copying to a portable mirrored TB3 RAID with SSDs may be a good solution to combine speed, redundancy, and reliability.
    Reply With Quote  
     

  7. #7  
    Build a super fast and affordable SSD raid which you can later charge half its value per day as a box kit item for DataMgmt. Offload cards 5x faster and have them back on set where they belong.
    Reply With Quote  
     

  8. #8  
    Now that you guys are talking about offloading, I was wondering what kind of transfer speeds you guys normally get when offloading 4k raw files. I often shoot in raw 4:1 with my Ursa Mini 4k, and offload with a Lexar CR1 via USB 3.0 to either an SSD in my desktop, my WD Raid 0 HDD external, or a standard 7200 rpm HDD internal. My transfer speeds are always around 30 MB/s regardless of the storage type I'm using. I've tried all of the different USB 3.0 ports in both my desktop and laptop and the speed didn't change. Also, I'm using Lexar 3400X 128 GB Cfast cards. I know the individual raw files should take longer to transfer than the Prores files, but it's taking on average about 10X longer, which sucks when I'm trying to transfer in the field. I truly appreciate any thoughts and considerations you guys might have.
    Reply With Quote  
     

  9. #9  
    Quote Originally Posted by Joe Grasso View Post
    Now that you guys are talking about offloading, I was wondering what kind of transfer speeds you guys normally get when offloading 4k raw files. I often shoot in raw 4:1 with my Ursa Mini 4k, and offload with a Lexar CR1 via USB 3.0 to either an SSD in my desktop, my WD Raid 0 HDD external, or a standard 7200 rpm HDD internal. My transfer speeds are always around 30 MB/s regardless of the storage type I'm using. I've tried all of the different USB 3.0 ports in both my desktop and laptop and the speed didn't change. Also, I'm using Lexar 3400X 128 GB Cfast cards. I know the individual raw files should take longer to transfer than the Prores files, but it's taking on average about 10X longer, which sucks when I'm trying to transfer in the field. I truly appreciate any thoughts and considerations you guys might have.
    The kind of files shouldn't make a difference in a large sequential transfer. To really diagnose though, we need more information. What software are you using to copy the files? Are you positive that your ports aren't defaulting back to USB2 because the cables are either malfunctioning or not inserted all the way? etc.
    Reply With Quote  
     

  10. #10  
    Quote Originally Posted by Alex.Mitchell View Post
    The kind of files shouldn't make a difference in a large sequential transfer. To really diagnose though, we need more information. What software are you using to copy the files? Are you positive that your ports aren't defaulting back to USB2 because the cables are either malfunctioning or not inserted all the way? etc.
    Thanks for the response, Alex. I'm just using Windows explorer. I open the folders in Explorer, select all of the files (usually a combination of Prores and raw clips). All of the Prores clips copy over as fast as I would expect...in the 250-400 MB/s range, when it gets to the raw files, it goes all the way down to 30 MB/S and even less. I'm using Windows 10.
    Reply With Quote  
     

Similar Threads

  1. Fastest way to off load CinemaDNG Raw file from a Cfast card.
    By amsh89es335 in forum General Discussion
    Replies: 1
    Last Post: 01-10-2017, 11:54 AM
  2. Replies: 9
    Last Post: 09-25-2015, 02:22 AM
  3. Replies: 10
    Last Post: 08-01-2013, 09:17 PM
  4. Simple Question- Fastest way to transcode 4TB of Raw footage?
    By SamuelJacobPauling in forum Workflow
    Replies: 8
    Last Post: 05-26-2013, 11:37 PM
  5. Playback of raw DNG after offload
    By Martin in forum General Discussion
    Replies: 5
    Last Post: 09-10-2012, 10:48 AM
Bookmarks
Bookmarks
Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •