S3, or Simple Storage Service is Amazon’s high capacity, high resiliency object store. The company recently announced a new feature, which allows versioned objects (those objects where multiple versions are retained for historical archive purposes) in S3 to be managed with lifecycle rules that either move data to AWS Glacier or assign a time period for the deletion of historical copies.
Versioning is a feature of S3 which allows multiple copies of an updated object to be retained, based on policies set by the user. Policies are established using AWS Rules, which can be applied to an entire bucket (a collection of objects) or a subset based on a folder or object prefix. From this point the user defines an action on either the primary or the versioned copies of the object. In many instances the primary object may be left alone (“do nothing”) whereas the older versions get archived or deleted after a fixed period of time. Archiving means moving to Glacier, Amazon’s low cost storage.
Although the Lifecycle rules a pretty simplistic, they are actually quite powerful and offload a lot of work customers would have done themselves. As the objects never leave S3/Glacier, all of their metadata information should be maintained automatically.
How could this feature be used? Well think of something as simple as backup; a lightweight backup client could replicate into S3 with versioning enabled with rules/policies in place to move all but the primary version immediately into Glacier. Even backups taken with something as simple as rsync could be made to be cost effective and more useful.
The Architect’s View
Amazon continues to innovate with AWS, to the extent that it’s almost a full time job keeping up with the enhancements. Versioning of objects and archiving to Glacier is a “must have” feature that will make S3 even more attractive to the customers already using the platform and perhaps help to jump start the use of Glacier.
- Amazon Delivers Cloud Archive Storage with Glacier
- Amazon S3 Lifecycle Management for Versioned Objects (AWS Blog, 20 May 2014)
Comments are always welcome; please indicate if you work for a vendor as it’s only fair. If you have any related links of interest, please feel free to add them as a comment for consideration.
Subscribe to the newsletter! – simply follow this link and enter your basic details (email addresses not shared with any other site).
Copyright (c) 2009-2014 – Chris M Evans, first published on http://blog.architecting.it, do not reproduce without permission.