Diff: BashHashes

Differences between version 2 and previous revision of BashHashes.

Other diffs: Previous Major Revision, Previous Author

version 2

Greetings

Bash 4.x has associative arrays (hashes if you are an old perl user like me). The existing examples of how to use them on the internet are not very well documented, so I thought I would write up my experience from a recent real world project.

Note this only works for bash 4.x.

First Example

Use a while/read loop to process a file line by line. Here is a sample input file (file.txt) which comes from the aws cli 'describe-snapshots' output (in text mode). The fields are tab-delimited.

SNAPSHOTS       service backup 2016-09-06   False           742630462602    100%    snap-13456582   2016-09-06T18:19:13.000Z        completed       vol-b13fc255    1024
SNAPSHOTS       service backup 2016-09-07   False           742630462602    100%    snap-54938492   2016-09-07T00:00:09.000Z        completed       vol-b13fc255    1024
SNAPSHOTS       service backup 2016-09-08   False           742630462602    100%    snap-32949585   2016-09-08T00:00:09.000Z        completed       vol-b13fc255    1024

we need to retrieve the snapshot ids and the date each was created, so that we can determine if a snapshot is old enough to be deleted.

# assoc array to hold the results
declare -A snapIds

# read the file and extract id and date for each line
while read line; do
  snapId=$(echo "${line}" | cut -f6)
  # we don't care about the specific time, just the date
  date=$(echo "${line}" | cut -f7 | cut -dT -f1)
  # create hash entry with key of snapId and value of date
  snapIds["${snapId}"]="${date}"
done < file.txt

Second Example

Same as example one, but process the contents of a multiline variable, instead of reading a file, by use of a bash here-string:

# assoc array to hold the results
declare -A snapIds

# Get all the matching snapshots from aws
snapshotDump=$(aws ec2 describe-snapshots --region us-west-2 \
                                --filters "Name=description,Values=service backup*" \
                                --output text | grep "^SNAPSHOTS")

# read the file and extract id and date for each line
while read line; do
  snapId=$(echo "${line}" | cut -f6)
  # we don't care about the specific time, just the date
  date=$(echo "${line}" | cut -f7 | cut -dT -f1)
  # create hash entry with key of snapId and value of date
  snapIds["${snapId}"]="${date}"
done <<< ${snapshotDump}

Manipulating Hashes

In both cases the result is the same, a hash of each snapshot id and the date it was created. We can view the results like so:

for i in "${!snapIds[@]}"; do
  echo "id: $i date: ${snapIds[$i]}"
done

Some things to note here:

• you always have to reference hashes inside ${}
• ${!snapIds} (with the !) gets you the keys. Without the !, you get the values.
• similar to bash arrays, ${!snapIds[@]} give you all the keys. Thus, ${snapIds[@]} gets all the values.

Now you could use some code like this to run through the hah keys and do something with each value:

# determine epoch date for one year ago:
oneYearAgo=$(date -d "1 year ago" +%s)

for snapId in "${!snapIds[@]}"; do
  date=${snapIds["${snapId}"]}
  # convert date to epoch time
  epochDate=$(date --date="${date}" +%s)
  if [ $epochDate -lt $oneYearAgo ]; then
    # snapshot is more than one year old, delete it.
    aws ec2 delete-snapshot --snapshot-id "${snapId}" --region us-west-2
  fi
done

Conclusion

I hope you've enjoyed these examples of how to use some more advanced bash 4.x features. I know it may seem silly to do all this work in bash, but the facilities are there if you want to use them.