Beware: The Classic And The Elastic Resize Of AWS Redshift Are Fundamentally Different Operations

  • The concern is documenting the 2 fundamentally different ways to resize AWS Redshift.
  • the point being made is that the names (classic and elastic) are not suggesting the amount of diff
  • yes, elastic is new, but it is for different usecases, has different timescale and pricing — took time to be to grasp the nature of the relationship between the two

1. classic

  • usecase: you need to add just a partial amount of space, a one or two nodes to the existing cluster
  • data is copied to a **BRAND NEW CLUSTER **
  • the source cluster is READ-ONLY
  • Classic resize takes 2 HOURS–2 DAYS or longer, depending on the data’s size

2. elastic

  • same cluster
  • during the operation, the cluster is read-only
  • takes 10–15 MINUTES
  • really different logic, the name may be misleading
  • Before deciding whether elastic resize is appropriate consider constraints
  • The new node configuration must have enough storage for existing data.
  • Even when you add nodes, your new configuration might not have enough storage
  • Why ? Because of the way that data is redistributed.
  • You can resize only by a factor of 2, up or down, for node types.
  • For example, you can resize a FOUR-NODE cluster up to EIGHT NODES or down to two nodes.
  • This limitation exists to avoid data skew between nodes caused by an uneven distribution of slices.
  • For node types
  • you can resize up to two times the original node count, or down to one-half the original node count.
  • For example, you can resize a 16-node cluster to any size up to 32 nodes, or any size down to 8 nodes. This limitation exists to avoid data skew between nodes caused by an uneven distribution of slices.

3. why would you want elastic resize

  • you need one /combination of those
  • fast/immediate change (both up & downscale)
  • temporary change
  • available service (no downtime)
  • lots of compute power
  • lots of storage
  • usecase: you need a deep copy of really large tables

Sources

Written by

today I learnt… | as a support eng of the wonderful Snowplow Analytics, expect everything around modern (postmodern?) business intelligence

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store