Home | Storage | Understanding Tiered Storage
Understanding Tiered Storage

Understanding Tiered Storage

0 Flares Twitter 0 Facebook 0 Google+ 0 StumbleUpon 0 Buffer 0 LinkedIn 0 Filament.io 0 Flares ×

This week I noticed a conversation taking place on Twitter discussing the subject of tiered storage.  The understanding of why tiered storage developed and why it is used today seems to be somewhat misunderstood, so perhaps it’s time for an update on this technology.


Given infinite resources and infinite budget, all data would be stored on the fastest media possible, because we know that storage is the major bottleneck in most IT infrastructure.  Unfortunately life isn’t that simple; businesses have budgets and are constrained in environmentals (floor space, power, cooling), plus staffing resources and of course with exponential growth, hardware costs grow with a similar profile.

Tiered storage developed to fix the problems of escalating storage costs, because storage demand grows anywhere from 50-100 per year and even with declining disk prices, a single tier model is unsustainable.

The Long Tail

Now, when we look at data on shared storage, we see the Long Tail effect – a small number of LUNs or volumes consume the majority of I/O.  So from a performance perspective, not all data needs to be treated the same way.  In fact, in general with large shared storage environments, 80% of the IOPS consumed comes from 20% of the data, although this will vary slightly from environment to environment.  The opportunity therefore with tiered storage is to place data on the tier of storage that delivers the performance it requires.  To achieve this we need to look at storage I/O density.

Storage I/O Density

I/O Density is the measure of how many IOPS a tier of storage can deliver per unit of capacity.  It can be measured, for example in IOPS/GB or IOPS/TB.  15K FC drives therefore provide a higher I/O density than the same capacity 10K drive and flash drives provide significantly higher levels than spinning media.  Applications typically have their own I/O density requirements, give or take some burst capacity for busy periods.  Designing tiering should therefore be about building tiers that deliver to a specific I/O density profile.

Tiering 1.0

The first implementations of tiering were pretty basic and worked at the granularity of the array.  Assuming you could justify multiple purchases, each array would be packed with data that mapped to a certain level of performance, for example, 15K FC drives, 10K FC drives or SATA drives.  The performance figures of these tiers could be varied by implementing different RAID types with differing RAID stripe widths (taking into consideration data availability requirements of course).   Moving data between tiers in the 1.0 model was cumbersome, requiring host-based migration or array-based replication and in most cases this type of tiering was in place before features such as Storage vMotion.

Tiering 2.0

The second wave of tiering saw multiple tiers of storage deployed in the same array.  Each tier presented LUNs/volumes built from that tier of disk.  Although this model was more efficient (no need to buy multiple arrays), there was still the risk of wasted resources if the storage was purchased in the wrong ratio, or demand for each tier type wasn’t well understood.  This created the dilemma of deciding whether to fully fill an array or to be continually upgrading it with new disk.  Data mobility between tiers was slightly better, as tools were available to automate this in the background on some platforms, but all of the data on that volume/LUN would get the same level of performance, which could still be inefficient.

Tiering 3.0

The third wave of tiering saw data being divided up at the sub-LUN/volume level, with arrays determining “hot” or active data and moving it to faster storage, while inactive data moves to lower performing disk.  These solutions become more efficient in terms of resource usage but don’t have the flexibility to react to changes in I/O demand as data is moved infrequently to reduce the change of contention or “churn” where more data is moved around than necessary, having an impact on production performance.  Of course no wholesale migration of data is necessary, only hot/cold blocks are moved.

Tiering 4.0

The latest wave of technology doesn’t look to place data on specific tiers of storage.  Instead, storage architectures have been rewritten to take advantage of high-performance flash devices and use them as read and write cache.  This approach is different from previous implementations that placed data permanently on fast storage as data on flash can be more transient.  Flash is such a fast I/O technology that in many instances flash performance isn’t fully utilised.  These new storage designs allow flash to be fully exploited, creating a more attractive $/GB ratio and allowing more dynamic placement of active and inactive data.

Exceptions to The Rule

Of course there will always be exceptions to these general rules, however they are still forms of tiering.  Some applications need low latency I/O and therefore justify dedicated hardware.  This may be in the form of a dedicated appliance, or hardware in the server such as NVDIMM or PCIe SSD.  Some data such as backup and archive have an I/O profile that creates sequential I/O and can be delivered through high capacity, low (random) performance disk drives.  We’re also seeing a move to new application models that distribute the I/O across many servers, allowing relatively inexpensive disks to provide large numbers of aggregated I/Os.

The Architect’s View

The last 10 years have seen an evolution in the placement of data to best exploit hardware resources as efficiently as possible.  However the high-end, all encompassing storage array has had its day as new application deployment models emerge.  Tiering will still exist and the basic principles of I/O density, latency and performance will all still apply.  Ultimately it’s all about optimising cost, regardless of the application architecture being used.


Comments are always welcome; please indicate if you work for a vendor as it’s only fair.  If you have any related links of interest, please feel free to add them as a comment for consideration.  

Subscribe to the newsletter! – simply follow this link and enter your basic details (email addresses not shared with any other site).

Copyright (c) 2009-2014 – Chris M Evans, first published on http://blog.architecting.it, do not reproduce without permission.


About Chris M Evans

Chris M Evans has worked in the technology industry since 1987, starting as a systems programmer on the IBM mainframe platform, while retaining an interest in storage. After working abroad, he co-founded an Internet-based music distribution company during the .com era, returning to consultancy in the new millennium. In 2009 Chris co-founded Langton Blue Ltd (www.langtonblue.com), a boutique consultancy firm focused on delivering business benefit through efficient technology deployments. Chris writes a popular blog at http://blog.architecting.it, attends many conferences and invitation-only events and can be found providing regular industry contributions through Twitter (@chrismevans) and other social media outlets.
  • Pingback: Cloud as a Tier of Storage | Architecting IT Blog()

  • Tim Chung

    It looks like the Tiering 3.0 in this article is the
    (auto) Tiering and 4.0 is the (SSD) Caching. On Qsan AegisSAN we offer SSD read
    caching which can speed on the read performance for the “hot data”. The big
    difference between Tiering and Caching is that the total storage capacity is
    the sum of all tier capacities, no matter it is SSD, SAS, NL-SAS, or SATA
    disks; but in Caching, the SSD cache size is not counted into the total storage
    capacities, just like you will not see the memory cache on the controller as
    part of the storage capacities.

    No matter it is Tiering or Caching, both need the
    “warm up” time to filter out the hot data and move (Tiering)/copy (Caching)
    into the SSD or upper tier to speed up the I/O. The warm up timeframe for
    Tiering and Caching can vary, say Caching warm up timeframe is usually from
    minutes to hours; whereas the Tiering warm up timeframe takes from days to more
    than a week. This major difference on warm up time makes Caching is suitable
    for short-term changing workload on a daily or weekly basis and Tiering is
    suitable for long-term changing workload on a monthly/quarterly basis.

    Tim Chung, Qsan Technology (www.qsantechnology.com)

0 Flares Twitter 0 Facebook 0 Google+ 0 StumbleUpon 0 Buffer 0 LinkedIn 0 Filament.io 0 Flares ×