External Storage Hosting with BackBlaze ($0.005 per GB)

[RyanTranquil]

Not sure if you're familar with BackBlaze but they now offer their new B2 service which is the cheapest external storage method on the planet (beating AWS, etc).. with a rate of $0.005 per GB. Link = https://www.backblaze.com/b2/cloud-storage.html

Would this be an external platform that Chevereto would consider adding to their implementation list in future versions? If not, our team plans to custom implement it soon.

Thanks,
Ryan

 

[Rodolfo]

Sure, I'm just waiting for an official PHP SKD.

 

[malvernweb]

I fell out of favour with the big storage companies.

As part of my "day job" I have need to store over 200tb of data. It has nothing to do with image hosting, but the principal is the same. The problem with big data companies is the hidden charges. OK, you get to store data for fractions of a $, however, it's not about the storage, its about the serving of the data.

So you use S3, B2, Rackspace Cloud (etc) and you get to store data, cheaply. But then somebody requests one of your pages, and you get charged for it. They then serve your page, and you get charged for the bandwidth. For little sites, OK this might not be the end of the world, however, the bill for bigger sites can get silly in PUT/GET and bandwidth bills. I was paying 4x more for the bandwidth and operations than I was for the storage each month.

We (in my other company) dropped S3 altogether and built our own storage cluster with OpenStack and Swift. Same redundancy, same infinite storage, but no bandwidth and request costs. That setup cost an initial $40,000, which OK isn't practical for startup image sites, however if you look at the other options for CHV sites, there are better IMO.

If you take a $16/m OVH server, with a 2TB hard drive. Load it with NGINX and use it as an FTP server, you are getting 2TB for your $16, but no bandwidth charges, no request charges and a dedicated datapipe that you can use to your hearts content and know that the bill for that server is only ever going to be $16.

S3 etc are great for archiving, but IMO, not so great for serving files. A lot of the big companies (Dropbox, Netflix etc) are starting to do the same and build there own. I heard rumor that Netflix were paying $2-4m/m, just for bandwidth .... its crazy.

 

[Rodolfo]

Just a side note, try to use SFTP instead of FTP. It handles better and faster.

 

[SirMoo]

I heard rumor that Netflix were paying $2-4m/m, just for bandwidth .... its crazy.

Give that they're the number one down provider of data in the US... I don't find that number all that suprising given the size of hd content... They killed their own network for a hosted solution elsewhere which likely saves them money in terms of manhours.

 

[lovedigit]

So you use S3, B2, Rackspace Cloud (etc) and you get to store data, cheaply. But then somebody requests one of your pages, and you get charged for it. They then serve your page, and you get charged for the bandwidth. For little sites, OK this might not be the end of the world, however, the bill for bigger sites can get silly in PUT/GET and bandwidth bills. I was paying 4x more for the bandwidth and operations than I was for the storage each month.

If you take a $16/m OVH server, with a 2TB hard drive. Load it with NGINX and use it as an FTP server, you are getting 2TB for your $16, but no bandwidth charges, no request charges and a dedicated datapipe that you can use to your hearts content and know that the bill for that server is only ever going to be $16.

S3 etc are great for archiving, but IMO, not so great for serving files.

That's exactly what I experience with my underlying infrastructure for image hosting provided by Amazon Web services. The S3 storage is insanely cheap, it is actually a great service. However, GET, PUT request + Bandwidth charges make no sense. During my first month of the image hosting, I was charged $0.03 for S3 storage and $49 for bandwidth. I was like, "Are you fucking kidding me?"

Now I use a VPS with 3TB dedicated bandwidth, with 600Mbps guaranteed line, during my test I found that usually it's 1800Mbps. On top of all, it costs only $10/month with 250GB storage with 480Mb/s IO and it is expandable up to 10TB. I couldn't ask more. I can now focus on actual work rather than having to pay for silly bandwidth charges. I wouldn't recommend anyone to use object storage such as Amazon S3, Backblaze, or other third-party service providers unless your customers are paying for the service.

 

[malvernweb]

If there's ever a worry that using a set of single disk (s)FTP servers isn't going to provide the fault tolerance that paid object storage is, you can always set up incremental backups using S3cmd, or the equivalent to run weekly/nightly/hourly backups to the big storage providers. This way you get the convenience of their resilient, fault tolerant storage, but without the bills they accumulate for their bandwidth and requests. Incoming bandwidth tends to be free, so the storage would be all you use. If you are smart and copy your config files as well (Apache, users etc) In the event of FTP server failure and, you can just grab a new one, download all the corresponding data to it straight from the storage provider (including all your configs), update your DNS records and be back up and running in under an hour, even for many TB's of data.

 

[lovedigit]

If there's ever a worry that using a set of single disk (s)FTP servers isn't going to provide the fault tolerance that paid object storage is, you can always set up incremental backups using S3cmd, or the equivalent to run weekly/nightly/hourly backups to the big storage providers. This way you get the convenience of their resilient, fault tolerant storage, but without the bills they accumulate for their bandwidth and requests. Incoming bandwidth tends to be free, so the storage would be all you use. If you are smart and copy your config files as well (Apache, users etc) In the event of FTP server failure and, you can just grab a new one, download all the corresponding data to it straight from the storage provider (including all your configs), update your DNS records and be back up and running in under an hour, even for many TB's of data.

Yeah, I am using both Backblaze and Amazon S3 for backup. In case one of them ever go AWhol. You never know.