Home | Storage | Moving On From Storage Tiering
Moving On From Storage Tiering

Moving On From Storage Tiering

0 Flares Twitter 0 Facebook 0 Google+ 0 StumbleUpon 0 Buffer 0 LinkedIn 0 Filament.io 0 Flares ×

One of the subjects that was raised again at last week’s  A3 IT Question Time in London was that of storage tiering.  To recap, the concept is pretty straightforward; you install storage within an array or architecture that has multiple levels of performance and data is placed on the appropriate tier based on its performance requirements.  In more complex environments, data can be dynamically moved between tiers as required.  However I believe the tiering model as it exists today has come to the end of its useful life.  The reasons for this include:

  • Rigidity – Tiering is a fixed model.  You install categories of storage in the array and your server gets whatever performance level that tier can provide.  However each of those tiers are fixed steps (for example the difference between 15K & 10K drives) which means inevitably, some hosts are receiving more IOPS capability than they demand, and that’s wasted resource.  Dynamic tiering helps with this but of course is typically implemented over a measurement period in days rather than seconds or milliseconds.  If a workload profile changes, adding “adding more IOPS” means physically installing and de-installing disks.  This issue can be mitigated somewhat by adding in disk to match the workload profile as part of a growth plan, but that still means effort in planning and internal data migration.
  • Waste – To achieve dynamic tiering means having spare capacity available to promote or demote data.  This immediately means disk space is being under-utilised just in case a requirement to move data arises.  Creating many levels of tiering also creates waste because it is difficult to guess in advance what amounts of each tier will be required at array deployment time.
  • Complexity – Tiering models are just too complex to manually interact with.  As environments scale and change dynamically, the tiering model needs to be reviewed frequently, even when dynamic/automated tiering is implemented.  Having many tiers across multiple architecture from multiple vendors means a lot of work in getting a consistent architecture.


Best Efforts

Essentially, almost all storage arrays today work on the idea of best efforts.  All I/O is delivered as fast as possible, with algorithms and queuing techniques used to improve throughput.  This shouldn’t be a surprise, because data was always stored on an unpredictable persistent medium that was orders of magnitude slower than the compute and memory components – a medium known as the hard disk drive.  When I/O response time from disk is unpredictable, there’s no way to implement a quality of service algorithm that can guarantee response time.  The only way to reduce latency is to keep more data in cache and have a big cache, but that then creates issues of data integrity, scale out/up and power consumption.  Think of all those heavy duty batteries deployed today just to protect the cache in the event of a power failure.  So with physical disk, best efforts was all we could expect.


Infinite Tiers

The ideal solution, excluding the issue of capacity, is to have a scenario that allows for an infinite number of tiers.  As the tier count increases, each server will be closer to receiving the performance level they need, with an infinite number of tiers delivering to every possible performance level.  If we can deliver that, move the data between tiers dynamically and fix the wasted capacity issue within each tier, then we have our ultimate storage device.  It may sound like a tall order, but its not really.


Storage Performance Hierarchy

We are lucky to have experienced an increase in the number of methods used to store data over the last few years.  The hierarchy now goes something like this:
  • DRAM – very, very fast, but volatile and expensive.  Not a great scale out solution.
  • Persistent DRAM – emerging technologies like that from Diablo, where NAND is build in the DIMM form-factor, accessible on the motherboard, with good scale and persistence.
  • Flash – in many guises – PCIe SSD, custom NAND cards, SLC SSD, MLC SSD.  Persistent storage with large capacity and scale.  Still a little expensive though.

All of the above provide low latency with predictable response times.  Then there are the hard drives.  These scale from 15K RPM, through 10K, 7.2K and 5.9K and increase in capacity as speeds reduce.  Perhaps with new recording techniques we’ll see even slower drives, with much larger capacities.

As we can see, we’re not just stuck with spinning disks.  There is the opportunity to build arrays from a range of devices offering a balance of performance versus capacity.

Putting it all Together

What we want in the future is an infinitely scaleable array that delivers I/O to a service level, rather than a fixed physical storage tier.  This means assigning latency and IOPS to each LUN.  Even at the simplest level, we have the ability to use our DRAM and Flash layers to stage reads and writes so they are delivered to the host with the guaranteed service level assigned to the data.  For example, imagine providing a guaranteed 5ms response on I/O write to a host.  In the traditional model, once the data is in non-volatile cache, the I/O would be acknowledged and de-staged to disk/flash.  But why do the acknowledgement immediately?  Why not hold it until the 5ms requirement and do it then?  There are some issues to be resolved around data integrity in this model, but those exist already today and are catered for by the array.

Delivering to a service level already exists; SolidFire, a vendor I’ve mentioned many times, already does this.  However at the moment, their solution is based purely on Flash, because, unsurprisingly, Flash gives the most predictable performance while delivering high numbers of IOPS.  The evolution from SolidFire is to build a system that knows how many IOPS can be delivered from any underlying hardware (adding up disk, flash, DRAM) and match that to the IOPS requirement per LUN.  This would truly be Software Defined Storage (SDS).

The Architect’s View

Our array vendors may already be working on the next generation of storage that really does deliver to SDS and quality of service.  I’m hoping that there’s promise in solutions like ScaleIO to deliver to the infinite tiers model.  When we get to that point, our work in the physical storing of data will be pretty much done, and we can all move onto the next stage of getting more value out of all of this stuff we’re keeping.

Related Links


Comments are always welcome; please indicate if you work for a vendor as it’s only fair.  If you have any related links of interest, please feel free to add them as a comment for consideration.

Subscribe to the newsletter! – simply follow this link and enter your basic details (email addresses not shared with any other site).

Copyright (c) 2013 – Brookend Ltd, first published on http://architecting.it, do not reproduce without permission.

About Chris M Evans

Chris M Evans has worked in the technology industry since 1987, starting as a systems programmer on the IBM mainframe platform, while retaining an interest in storage. After working abroad, he co-founded an Internet-based music distribution company during the .com era, returning to consultancy in the new millennium. In 2009 Chris co-founded Langton Blue Ltd (www.langtonblue.com), a boutique consultancy firm focused on delivering business benefit through efficient technology deployments. Chris writes a popular blog at http://blog.architecting.it, attends many conferences and invitation-only events and can be found providing regular industry contributions through Twitter (@chrismevans) and other social media outlets.
  • Augie Gonzalez

    Hi Chris, Confining tiering within an array may well lead to the shortcomings you point out. The company I work for, DataCore, extends sub-LUN auto-tiering across a multiplicity of storage devices ranging from flash, SSDs, DAS, external arrays and public cloud storage. We support 15 tiers but in practice find 3 or 4 prove quite suitable. The device-independent storage virtualization software also takes advantage of mirrored DRAM caches (multi-cast stable storage) to improve on the SLA possible with the underlying persistent storage. Have a look at us when you have a chance http://www.datacore.com/Software/Features/List-of-Features/Auto-Tiering.aspx

  • Ilja Coolen

    Hi Chris,

    Unlimited tiers seems somewhat much, but I guess with variable speed spindles instead of fixed speeds this would be a technical solution to your service level based approach. Combined with flash technologies of course. The array would decide what speed would be suitable for a given workload, and put all the chunks of data from various matching workloads on that spindle (with proper failure protection obviously).
    That would make is possible to have close to unlimited tiers without having the array manage many disparate types of devices. I think this would benefit reliability and predictability.

    Delaying a write ACK to the agreed upon service level seems strange though. I think the biggest problem here lies with customers or application owners inability to quantify their requirements.

  • http://www.vuemuer.it/ Luca Dell’Oca

    Nice article Chris.

    About software only storage with QoS, I’ve read something about Cloudbyte for example and they seems to do what you are willing to see: software only solution, can offer QoS by knowing the underlying hardware they are installed on. Not sure how they can guarantee IOPS and QoS without having control on the hardware, maybe they “learn” the available performances once it’s installed, and adjust QoS. They will present at SFD4 in November, I hope to find out more…

    Otherwise, I’m with Solidfire in the idea to control the performances by offering software AND hardware in order to have full control. I’ve seen in the past many customers doing bad design with storage VSAs and then blaming the software….


  • Bryan Wagoner

    Seems like concentrating on Tiering as a single feature is too one-dimensional when describing modern arrays. Remember that arrays offer both sub-lun tiering and SSD l2 cache at the same time to service long term and also dynamic workloads.. Also , many are starting to offer primary dedup and compression which will both enhance density of the few tiers that exist but also create the ability to make tiers-within-tiers as luns or even individual apps can be applied with dedup or varying compression levels. Compression and dedup can even enhance performance in some workloads. Combine this with Cloud and locality architectures which implements ssd caches on servers and also the fact that memory on servers continues to become cheaper and more dense so I/O cache on primary serve memory is also on the increase and you already have a whole lot of tiers in today’s end-to-end I/O architectures. Then there’s app level technologies like HCC in Oracle that can be taken into consideration to improve storage properties. I just used HCC in Oracle as an example, but I meant application enhancements at the application level could be used to solve the problem rather than doing it in hardware. Continuing to increase hardware tiers seems like unnecessary complexity when simplification should be the ultimate goals of architectures.

  • Pingback: Moving On From Storage Tiering()

  • http://www.zadarastorage.com/ Zadara Storage

    Hi Chris,

    Actually we at Zadara Storage are not just working on, but DELIVERING SDS with QoS today, for on- and off-premise cloud customers. Our Virtual Private Storage Arrays (VPSA) are scaleable, and deliver I/O to a service level. Every volume has 4 QoS controls: Read IOPS, Write IOPS, Read Throughput, and Write Throughput.

    An earlier blog post I wrote outlines our approach to QoS, which we think combines the best of common current approaches – from brute force, to dedicated drives and SSD caching: http://blog.zadarastorage.com/2013/04/cloud-qos.html

    As much as we dislike the hype around SDS as a term, I think our solution is a strong example of what SDS is, and the real benefits next-generation storage should offer.

    – Noam

    Noam Shendar
    VP Business Development, Zadara Storage

0 Flares Twitter 0 Facebook 0 Google+ 0 StumbleUpon 0 Buffer 0 LinkedIn 0 Filament.io 0 Flares ×