Tag: AWS

AWS Lambda RDS Snapshot Copies – Long term backups

AWS’s managed database service is pretty awesome. Its a managed service that takes care of the heavy lifting for managing a database server letting you focus on running your application rather than managing a database server. The service provides patching, automated backups etc.

A nice feature of the AWS RDS service is automated backups. The service will create automated backups on the schedule you desire (daily, weekly etc.) and gives you up to 35 days of backup retention. But what if you need more than 35 days of database backups?

Do you create your own backups manually? Ah, nope. Manually means its not going to get done. Do you schedule native backups? Ah, while somewhat possible, its not very elegant. How about using Lambda? Yes! Serverless code. But how?

Using this python lambda script and the AWS python API’s you can copy the current latest RDS snapshot and maintain a weekly retention history that suits your needs. Likewise, you also might need to retain a long term history of monthly backups. We got you covered with that as well.

Here is a snippet of the documentation that describes the scripts functionality:

rds-copy-snapshots-lambda: Makes a copy of the most recent auto snapshot and deletes ones older than the set retention period.

There are two versions of the script: a weekly version and a monthly version. You can choose to use only weekly, monthly, or both of them as you see fit.

Weekly and Monthly Snapshots are named and tagged differently to allow for filtering by type as well as to prevent inadvertent deletion.

More details about the rds-copy-snapshots-lambda function can be found here: https://github.com/swmacdonald/rds-copy-snapshots-lambda

AWS S3 CP Command Line Example to Change Encryption keys

The AWS Console allows you to upload files to a bucket and set the server side encryption.  This defaults to AES256.  However, there are times when you may want to encrypt a S3 object using a specific key.  For example, RDS SQL database native restore.   Therefore at times you may need to change the key used to encrypt a S3 object.  In that case, this command line example may prove useful to others.

To change an existing AES256 encrypted object to another encryption key in KMS use the AWS S3 CP command:

aws s3 cp s3://mybucket/test.txt s3://mybucket/test2.txt --sse aws:kms --sse-kms-key-id <key arn>

Because the original file was encrypted with default server side encryption of AES 256 it will automatically assume AES256 and decrypt the file as part of the copy process to re-encrypt with the new key.

 

How to: Launch a AWS Linux Instance using Encrypted Boot Volumes

Using encrypted boot volumes for AWS Linux Instances is not very well documented.   Here is a how to for launching a AWS Linux instance using encrypted boot volumes.

Note that encryption has a performance impact on the instance, and thus it will require a larger instance size to run.  You will not be able to get a t2.micro instance to boot as it does not have enough CPU performance.

The process is broken down into 4 steps

  1. Create a Role for the EC2 instance to access encryption keys
  2. Create the Encryption key
  3. Create a custom AMI using your Encryption key
  4. Launch the Instance using your custom AMI.

All the instances launched from the custom AMI will use the same encryption key.  Create another custom AMI if you need different encryption keys.

 

Step 1: Create a IAM role which will be used by the EC2 instance.  (Used to access encryption keys to boot etc.)

  • Sign in to the IAM console at https://console.aws.amazon.com/iam/.
  • In the navigation pane of the IAM console, click Roles, and then click Create New Role.
  • For Role name, type a role name that can help you identify the purpose of this role. Role names must be unique within your AWS account. After you type the name, click Next Step at the bottom of the page.

Important

Role names must be unique within an account. They are not distinguished by case, for example, you cannot create roles named both “PRODROLE” and “prodrole”.

  • Expand the AWS Service Roles section, and then select the Amazon EC2 click Select.
  • Click Next Step to review the role. Then click Create Role.

 

Step 2: Create a EBS encryption key to be used to encrypt the target <instance_name> volumes.

  • Sign in to the AWS Management Console and open the Amazon RDS console at https://console.aws.amazon.com/iam/.
  • In the top right corner of the Amazon RDS console, choose the region in which you want to create the encryption key
  • In the navigation pane, choose Encryption Keys.
  • Click Create Key.
  • Enter a name for the encryption key in the Alias field.  Example: <instance_name>_ebs_key
  • Enter a description for the encryption key. Example: EBS encryption key for the <instance_name>
  • Click Next Step
  • Add tag meta data
  • Click Next Step
  •  In the Key Administrators field, select the users and or roles who will have administrative rights to administer the key.
  • Click Next Step
  • In the Define Key Usage Permissions – This Account  Select the <instance_name> role you created above
  • Click Next Step
  • Review the Key Policy and if acceptable, click Finish. 

 

Step 3: Create a custom AMI based on the current Amazon Linux AMI which contains encrypted volumes/snapshots.

This will allow for encrypted volumes on the target instance.

  • Locate the latest Amazon Linux AMI in your region by attempting to launch a new EC2 instance.
  • From EC2 Console, Click Launch Instance. The latest current Amazon Linux AMI will be listed at the top of the Quick Start List. Copy the ami-xxxxxxx number.
  • From the EC2 Console, Select AMIs from the left navigator.
  • Paste in the copied ami-xxxxxx into the search filter
  • Choose Actions > Copy AMI, select the Destination region and check the Encryption checkbox option
  • For Master Key choose the EBS encryption key you created above and assigned to the target instance role you created in step 1.
  • Click Copy AMI.
  • The AMI is being copied to your account using the encrypted volumes/snapshots.

 

Step 4: Launch a Linux Instance using the custom AMI with your Encrypted EBS Volumes

  • Sign in to the AWS Management Console and open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
  • In the top right corner of the Amazon EC2 console, choose the region in which you want to create the new EC2 instance.
  • In the navigation pane, choose Instances.
  • Click the Launch Instance
  • Choose the custom Linux AMI you created in Step 3
  • Choose the instance type and click Next: Configure Instance Details
  • NOTE: Encrypted EBS volumes requires a larger instance size due to the encryption/decryption overhead.  A m3.large or larger is recommended.
  • Set the Network, Subnet-Availability Zone, and Auto-assign Public IP options as required.
  • For IAM role, choose the Instance Role you created in step 1 above.
  • Set the Shutdown behavior to Stop, and check the Protect against accidental termination box to enable termination protection on the instance.
  • Click Next: Add Storage to proceed to the next step.
  • Review the default EBS volume configuration of 8 GB root volume using general purpose SSD.  Adjust the volume size if needed and click Next: Add Tags.
  • Click Next: Configure Security Group.
  • Assign an existing Security Group or create a new Security Group as needed. Click Review and Launch.
  • Review the instance settings and Click Launch to launch the instance.
  • On the Select an existing ssh key pair or create a new key pair screen, either create a new ssh key or assign an existing key pair. Click Launch Instances to launch the new instance.
  • Wait for the new instance to launch and connect to instance to verify.