Amazon’s new and Intelligent S3 storage class:

What is S3?

Amazon Simple Storage Service (Amazon S3) is a global Infrastructure as a Service (IaaS) solution. Amazon S3 facilitates highly scalable, secured and low-latency data storage from the cloud.

Yet another storage class?

Prior to this, S3 objects had the following storage classes:

  1. Standard – For frequently accessed data.
  2. Standard-IA – For long-lived, infrequently accessed data.
  3. One Zone-IA – For long-lived, infrequently accessed, non-critical data.
  4. Glacier – For long-lived, infrequently accessed, archived critical data.

By default, the Standard storage class contains all S3 objects unless explicitly specified. Standard-Infrequent Access comes into picture when your object is infrequently accessed. Standard-IA has a lower storage cost associated with it. This may seem ideal initially, Standard offers a storage cost of $0.023 per GB for the first 50 TB data stored per month. In contrast, Standard-IA offers a meager charge of $0.0125 per GB for all the storage per month.

When juxtaposed, the choices seem fairly obvious, right?

Wrong.

The caveat with storing objects in the Standard-IA class is it’s considerably high data retrieval and request prices. Initially, you might store an object in Standard-IA under the assumption that you would barely access it, but as time progresses the need to use the object increases and along with it comes the high data request charges tied to the Standard-IA class. This would presumably lead to a spike in your bill. But, it is also not feasible to keep track of the usage patterns of objects and switch them between the storage classes frequently. Hence, using Standard-IA might end up being more than what you bargained for.

This is where the S3 Intelligent-Tiering storage class comes into play.

What is S3 Intelligent-Tiering Storage?

At re: Invent 2018 AWS introduced the Intelligent-Tiering Storage.

The S3 Intelligent-Tiering storage class is designed for customers who want to optimize their storage costs automatically when data access patterns change, without performance impact or operational overhead. The new storage class offers the same high durability, low latency, and high throughput of S3 Standard or Standard-IA and inherits all of the existing S3 features including security and access management, data life-cycle policies, cross-region replication, and event notifications.

How does it work?

Initially, the S3 objects are by default present in the Frequent Access tier for a period of 30 days. After monitoring data access patterns, objects that have not been accessed for a period of 30 days are moved to the Infrequent Access tier. Once accessed, they’re moved back to the Frequent Access tier.

How are Life-Cycle Policies affected by Intelligent-Tiering?

Source: https://docs.aws.amazon.com/AmazonS3/latest/dev/lifecycle-transition-general-considerations.html

  • The above waterfall model shows the possible storage class transitions supported by S3 life-cycle policies.
  • Transitions supported for Intelligent-Tiering are only to Glacier and OneZone_IA.
  • AWS does not transition objects that are smaller than 128kb to Intelligent-Tiering storage as it is not cost-effective.

Is this right for you?

  • If your data retrieval patterns are erratic and cannot be predetermined then the Intelligent Tiering storage class is for you, if not, the other four classes might be more suited.

What should you keep in mind before using Intelligent-Tiering?

  • This class should contain objects that are present for a minimum of 30 days. Hence, it is not suitable for short-lived objects. Rather, it is more suited for objects whose access patterns change over a longer period of time.
  • If the objects present are deleted, overwritten or transitioned to a different class before the minimum storage duration of 30 days there is a prorated charge per GB for the remaining days.
  • Objects smaller than 128kb could be stored in this storage class but it is not advisable to, because :
    1. Auto-Tiering is applicable only to objects of size greater than 128kb. Hence, the object will always be present in the Frequent Access tier irrespective of change in access patterns.
    2. The pricing for this storage will always be corresponding to that of the Frequent Access tier as it never moves to the Infrequent Access tier.

How to choose the Intelligent-Tiering storage class?

While uploading the object you can choose the Intelligent-Tiering storage class in the Set Properties step.

How is it priced?

(Pricing of US-East(N Virginia) considered in the above image. Pricing varies with region.)

From the above image, we can see that,

  • The billing for storage in the Frequent Access Tier is the same as S3 Standard.
  •  The billing for storage in the Infrequent Access is the same as S3 Standard-Infrequent Access.
  • There is a monthly fee of $0.0025 per 1,000 objects when using this storage class due to monitoring and automation.
  •  There is no cost associated while moving from Infrequent Tier to Frequent Tier and vice-versa.
  • The request pricing is as same as that of S3 Standard.