“The point is this: No matter how fat a pipe you have to the Internet, at some given amount of data, it’s going to be faster, cheaper, or both to use some manual method to ship data on some storage medium.”
I wrote that in September, 2011, in what remains (if I do say so myself) a fascinating and true look at the challenges in transferring really, really big amounts of data (aka “calculating the bandwidth of a station wagon full of tapes hurtling down the highway,” with a side look at the Avian Carrier Protocol, or sending data by carrier pigeon).
Well, it’s time to update it — and, actually, there was something back from 2009 I failed to include.
Let’s say you’re setting up a cloud storage service. Okay. How do you fill it up in the first place? If you’ve got terabytes or petabytes of data, getting it over to the new cloud storage service is going to be a hassle. And a time-consuming hassle at that — 166 days to move a terabyte of data over DSL, according to Amazon.
It turns out that Amazon announced in 2009, and Google duplicated this week, a service that enables you to burn all your data to a hard disk, and simply mail it to a facility, where elves from Amazon or Google will upload the data for you. (No word on whether Google sends the disk drives back to you afterwards, though the form does ask for a return mailing address; Amazon specifies that it returns the drive.) Amazon’s service is called AWS Import/Export, while Google’s is called Offline Disk Import for Google Cloud Storage.
The companies each charge $80 per disk drive for this service, which means it behooves you to use the biggest disk drive you can rather than a whole lot of commodity disks (a la Backblaze). Amazon has an upper limit of 16 TB, though it indicates it might be able to handle bigger ones on request; Google’s form lists varying total size options, ending at more than 400 TB.) Amazon also charges a $2.49 per hour upload fee; Google doesn’t mention such a charge. For the time being, at least, Google is limiting the service to U.S. users; while Amazon supports it in Singapore and Ireland as well as in several U.S. locations, the return address for the Ireland location must be in the European Union.
In addition, Google requires that you encrypt the data before sending. While this adds time and space and money, it adds an interesting wrinkle in terms of security. We now know that online services are regularly being monitored by the Feds, and are required to decrypt data based on court orders and other legal documents. But vendors such as Dropbox, which got busted for this (coincidentally) in 2011, pointed out that if the data was encrypted before it was uploaded to the service, there was nothing they could do in the Feds came calling. So if you’re newly worried about what the NSA might find in your data on the cloud, send it all encrypted on a disk drive, and Bob’s your uncle
While Amazon’s 2009 announcement didn’t create a stampede of providers offering a similar service, it will be interesting to see, in light of Google’s announcement, whether other providers will follow suit – particularly after the NSA revelations. (One wonders if Google’s timing was a coincidence.)
Google said the feature was experimental and that it might “rapidly evolve the feature,” which might break backwards compatibility, and that it couldn’t guarantee quality of service.
Meanwhile, a number of people want to know whether the shipping-data-on-disk services will also work the other way, for quicker restores.