Posts Tagged - CloudFormation

This AWS CloudFormation YAML template demonstrates how a Kinesis Data Stream stream can be implemented as Lambda Trigger in AWS CloudFormation.

Simply deploy the following template via the AWS CloudFormation console. In the designer the template looks like this:

Template:

Now we can gather the stream name from the CloudFormation stack outputs section and send a test event using the AWS CLI:

  aws kinesis put-record --stream-name <value> --data <value> --partition-key <value>

CLI Reference: https://docs.aws.amazon.com/cli/latest/reference/kinesis/put-record.html

The output should look like this:

Now you can check in the Lambda Console if the Lambda has been invoked and what has been written to the logs.

Read More

The following CloudFormation snippet shows how to use conditional properties in an CloudFormation template. The example configures one or two subnets in the VPCOptions section of an Elasticsearch domain, depending on whether a parameter called ZoneAwareness is set to true or not.

Parameters:
 ZoneAwareness:
      Type: String
      AllowedValues: [true, false]
      Default: true

Conditions:
  ZoneAwarenessTrue: !Equals [!Ref ZoneAwareness, true]

  ElasticsearchDomain:
    Type: AWS::Elasticsearch::Domain
    Properties:
      ...
      VPCOptions:
        SubnetIds: 
          -  !Ref SubnetA
          - Fn::If:
            - ZoneAwarenessTrue
            -  !Ref SubnetB
            - !Ref "AWS::NoValue"
        SecurityGroupIds:
          - !Ref SecurityGroup

Read more Pseudo Parameters Reference

Read More

Using the intrinsic function Ref on a Stack created within another Stack only gives you the Id of the referenced Stack. If you want to get the StackName which is generated automatically you have to do a combination of the intrinsic function Split and Select as follows:

!Select [1, !Split ["/", !Ref MyStack]]

This works since the Stack Id is structured as follows:

arn:aws:cloudformation:eu-west-1:*********:stack/test-nested-MyStack-R5E52GRQGVZH/8d90dd40-17a7-11ea-b079-02c18823f600

The statement splits the Stack Id by deleimiter “/” resulting a list which contains the StackName on index 1 which we then can select using the Select function.

Read More

Recently I had the following problem, a CloudWatch Alarm based on the Error-Metric of a critical Lambda Function occasionaly caused notifications.

The reason for the notifications was quickly found through a search in the lambda logs. The errors were caused by lambda timeouts. Since lambda timeouts are not critical in the utilised architecture I was looking for a way to ignore them in the CloudWatch Alarms.

The solution is called Metric Math.

Metric math enables you to query multiple CloudWatch metrics and use math expressions to create new time series based on these metrics.

Source: Using Metric Math by AWS

Herewith it is possible to create a new metric excluding the timeouts by subtracting timeouts from errors. By default there is no metric for timeouts within lambda functions. But this metric can be extracted with a simple Metric Filter applied to the loggroup of the respective lambda function:

Then a CloudWatch Alarm can be created with a mathematical expression:

In the CloudWatch console the result of the template looks like this:

Read More

Recently I have experimented a little with Kinesis Firehose. The result is this concise undocumented template which setup an Kinesis Firehose S3 Delivery Stream preprocessed by Lambda in AWS CloudFormation.

Read More

Announced at reinvent 2017 in a CloudFormation Deep Dive session for “early 2018” - now (almost at the end of 2018) the time has come. AWS has added a new and long awaited feature to CloudFormation that will help in many cases: “Drift detection”.

With this feature you can see in the CloudFormation console which stacks have been changed manually. Simply select the desired stack, select “Detect drift” under actions and confirm.

In AWS words: Drift detection lets you detect whether a stack’s actual configuration has been changed outside of CloudFormation. To detect drift on a stack, select the stack, and then select Detect drift for current stack from the Actions menu.
Read more in the official documentation.
Read more in the official blog.

Read More

Implement S3 Bucket Lambda triggers in AWS CloudFormation can be quite tricky because of very often circular dependencies or errors like “Unable to validate the following destination configurations” occur. But if you take notice of the following, working with S3 Lambda triggers in CloudFormation will be easier.

• First, you have to specify a name for the Bucket in the CloudFormation template, this allows you to create policies and permission without worrying about circular dependencies.
• Secondly, you have to add a DependsOn statement to the Bucket referencing the Lambda Permission, this helps you to fix “Unable to validate the following destination configurations” errors since the bucket will only get created if the Lambda Function and all necessary policies, roles and permissions are in place.

Below you will find a GitHub Gist with a working example which takes care of all tips mentioned above. In this example, created *.txt files are read from a bucket and then deleted.

Read more: https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-destination-s3/

Read More

Amazon MQ with ActiveMQ under the hood offers several different protocols, each with its own endpoint. Unfortunately, you can’t reference or output them directly in AWS CloudFormation. To make this possible I use the following variants in a Single Broker setup, where AMQBroker is a CloudFormation resource of type AWS::AmazonMQ::Broker:

and these in an active/standby setup for high availability:

Read More

Unfortunately, it is currently not possible to output or use the domain name / URL of an Amazon API gateway via Fn::GetAtt in AWS CloudFormation. Therefore I provide the following CloudFormation snippets that enables you to do exaclty this. The snippets use a resource called RestApi of type AWS::ApiGateway::RestApi and a resource Stage of type AWS::ApiGateway::Stage.

Read More