For example, let's say you need to store 100TB. Your storage array will be running a form of data protection (typically RAID5 or RAID6) which creates an overhead of 10-20%. Your array will then have 110TB-120TB of raw disk provisioned for your 100TB usable capacity. For important workloads, you'll want to have another array for disaster recovery (DR). This would be another 110TB-120TB. Finally, you would make backups of your primary array - this could be a further 150TB.
All of a sudden, your 100TB of data requires 390TB of infrastructure to support it. This creates challenges as you have to manage multiple arrays, backup software and hardware, test backups, deal with disk failures, test primary to DR restores, the list goes on and on.
Enter Irving S. Reed and Gustave Solomon with the Reed-Solomon Erasure Coding system. First introduced in the 1960's, Erasure Coding has been used to correct errors in commercial storage media such as CDs and DVDs and now is available in some object storage systems. By utilising 4th generation erasure coding, object storage systems can deliver your 100TB with as little as 170TB of raw infrastructure.
Let's dig in to how this is possible:
Challenge #1: Protection from site failures
In our example, we deploy two arrays at different sites to ensure data is still available if the primary site goes down. Each array is its own entity and the data is replicated, so both arrays store the same data.
With 4th generation Erasure Coding, an object storage system can be deployed as a single entity spread across multiple (typically three) sites. If pieces of the object storage system fail (i.e. a single site) the Erasure Coding provides enough resiliency to keep the system operational.
Challenge #2: Backing up
In our example, we back up the data to protect us from data corruption and a double array failure.
With 4th generation Erasure Coding data corruption can be identified and fixed on the fly. Because of the protection 4th generation Erasure Coding provides, most systems have at least 9x9's of predicted uptime (99.9999999%) which translates to 31.5 milliseconds of downtime per year.
I remember when I was first looking at getting a smart phone. Smart phones had been around for several years but I had held off. I kept questioning the move: "Would I really use the extra features? Would a smart phone really provide extra value in my life?" As you can probably guess, I took the plunge and now I can't imagine my life with out it.
I held off for too long when it came to getting a smart phone. I would encourage you to consider the same with your storage requirements: Are you missing out by not taking the plunge?