CloudFormation - AWS Study Guide (v2.0)

Infrastructure as Code (IaC)

Infrastructure as Code is a practice that takes writing custom scripts to automate tasks related to managing configurations, provisioning, and deployments to a new level. It leverages software development practices for infrastructure management.

IaC uses software development practices like version control, testing, and continuous integration/continuous deployment (CI/CD) pipelines for infrastructure. Examples of IaC tooling include CloudFormation, Terraform, Chef, and Puppet.

This practice allows for: * **Cost reduction:** Less time spent by people and effort. * **Speed:** Automation-enabled speed through fast execution with more visibility. * **Less risk:** Reduced chance of human error and manual misconfiguration, which increases reliability, decreases downtime, and improves security.

The process starts with a template, which acts as a blueprint of the desired state of infrastructure. This template is processed by tooling (e.g., CloudFormation), which parses it and determines the necessary infrastructure changes. The tooling then makes API calls to the targeted environment to provision or modify servers. It also calculates the delta between the current and desired states, allowing focus on the template's end state rather than procedural scripting or change calculations.

CloudFormation enables provisioning AWS resources using Infrastructure as Code (IaC). It provides a repeatable process for provisioning resources and works with most AWS services. A real-world example is automating the infrastructure-provisioning process for EC2 servers.

CloudFormation Fundamentals - Terminology

Understanding key terminology is critical for comprehending how CloudFormation operates.

A template is a JSON or YAML formatted text file that acts as a blueprint for building AWS resources. It is provided to the CloudFormation service as input and describes the desired end state of the AWS infrastructure. CloudFormation then orchestrates the provisioning of this infrastructure based on the template.

When CloudFormation executes a template, it creates a stack containing the resources defined in that template. A stack represents a collection of related resources managed as a single unit. In essence, the stack is a provisioned instance of the template. To update resources through CloudFormation, the stack must be updated with a new template.

Before updating a stack, a change set can be generated to preview how proposed changes will impact existing resources. This is crucial for live systems, as some modifications might delete or recreate resources, potentially causing data loss (e.g., renaming an RDS Instance creates a new one and deletes the old one). Change sets provide visibility into the actions CloudFormation will perform.

CloudFormation Template Anatomy - Top-Level Sections

CloudFormation templates are structured documents with several top-level sections, though only the Resources section is mandatory.

The template sections define different aspects of your infrastructure and its deployment.

AWSTemplateFormatVersion

Defines the CloudFormation template format version. This is optional. There is only one valid value at this stage: '2010-09-09'.

valid_value: '2010-09-09'

optional: Yes

Use Cases:

Defining template version

Description

A text string that describes the template. This section is optional.

optional: Yes

Use Cases:

Documenting template purpose

Metadata

A section for objects that provide additional information about the template. This is also optional.

optional: Yes

Use Cases:

Adding supplementary template information

Parameters

A section that allows you to pass values into your template at runtime, for example, at stack creation. This section is optional.

optional: Yes

Use Cases:

Customizing stack deployments without modifying the template file

Mappings

A set of keys and values which can operate as a lookup table. This section is optional. Mappings enable you to use an input value to determine another value, such as determining an AMI ID based on the region.

optional: Yes

Use Cases:

Dynamically selecting values based on conditions (e.g., region-specific AMIs)

Conditions

A section where you can place conditions on whether certain resources are created or properties defined. This section is optional.

optional: Yes

Use Cases:

Conditionally creating resources based on input parameters or other logic

Transform

Where you can specify transforms that CloudFormation will use to process your template. This is an optional section. For example, it can include external snippets into a template or specify a version of the Serverless Application Model (SAM).

optional: Yes

examples: ['AWS::Serverless (SAM version)', 'AWS::Include (include code snippets)']

Use Cases:

Utilizing macros or SAM transformations for template processing

Resources

This is the main section and the only required section in a CloudFormation template. It defines the stack resources and their properties.

required: Yes

Use Cases:

Declaring all AWS resources to be provisioned

Outputs

This section allows you to return values from your stack. These values can be imported to other stacks, returned to describe stack calls, or displayed on the console.

optional: Yes

Use Cases:

Sharing information between stacks (cross-stack references)
Displaying key resource attributes upon stack completion

CloudFormation Template Languages - JSON and YAML

CloudFormation templates can be written in either JSON (JavaScript Object Notation) or YAML (YAML Ain’t Markup Language), each with distinct characteristics.

YAML is a human-readable data serialization language and a superset of JSON, meaning JSON can be parsed with a YAML parser. It is often considered more readable than JSON due to its conciseness and use of indentation for structure, avoiding noisy curly brackets. YAML allows inline comments. Whitespaces and lines act as delimiters. It supports complex data types. **Key/Value Pair Example:** ```yaml name : Craig course : CloudFormation Deep Dive level: intermediate ``` **Objects (List with 2 elements each) Example:** ```yaml Department: - finance : 12 IT : 8 - pre-sales : 1 marketing : 8 ``` **Array Formats (Inline and Indented) Example:** ```yaml # Inline Array Instance: [t2.micro, t2.small, t2. large] # Indented Array Instance - t2. micro - t2.small - t2. large ``` **Key Takeaways for YAML:** * Indentation is a key part of YAML and represents relationships between layers of data. * Colons separate key/value pairs. * Dashes represent lists of items (single dash followed by a space). * Do not use tabs; use spaces for indentation. **CloudFormation Specific YAML Examples:** * `Type: AWS::EC2::SecurityGroup` * `Type: AWS::EC2::Instance` **Pros:** More compact than JSON, allows inline comments, used in tools like Ansible Playbooks, Docker Compose, Kubernetes, supports complex data types. **Cons:** May not appeal to the Javascript crowd.

JSON is a lightweight data interchange format used throughout AWS (e.g., CLI, IAM Policies). It is based on Javascript and resembles Java and C#. **Key Components:** * **Key/Value Pairs:** Contain data (e.g., `"name" : "Craig"`). * **CloudFormation Specific Examples:** `"Type" : "AWS::EC2::SecurityGroup"`, `"Type" : "AWS::EC2::Instance"` * **Commas:** Separate data (e.g., `"Firstname": "Craig", "LastName": "Arcuri"`). * **Curly Braces `{}`:** Hold objects. * **CloudFormation Example:** ```json { "Resources" : { "HelloBucket" : { "Type" : "AWS::S3::Bucket" } } } ``` * **Square Brackets `[]`:** Hold arrays. **Pros:** Resembles popular languages, widely used in AWS. **Cons:** Does NOT allow inline comments.

CloudFormation Template Sections - Advanced Configuration

Beyond the basic structure, CloudFormation templates offer advanced sections for dynamic configuration, reusability, and conditional logic.

These sections allow for more sophisticated and flexible infrastructure definitions.

Parameters

Parameters allow values to be passed into the template at stack creation or update time. A default parameter can be defined, along with a list of allowed values and a description.

example_yaml: ```yaml Parameters: InstanceTypeParameter: Type: String Default: t2.micro Allowed Values: - t2.micro - m1.small - m1.large Description: Enter t2.micro, m1.small, or m1 .large. ```

Use Cases:

Customizing instance types, database names, or other resource properties during deployment.

Mappings

Mappings match keys to corresponding name-value pairs, acting as a lookup table. They are useful for determining values like AMI IDs based on the region. The `Fn::FindInMap` intrinsic function is used to retrieve values from mappings. AMIs are region-specific, so ensure they are copied to the target region. **Example:** ```yaml Mappings: RegionMap: us-east-1: AMI: ami-76f0061f us-west-1: AMI: ami-655a0a20 Resources: Ec2Instance: Type: 'AWS::EC2::Instance’ Properties: ImageId: !FindInMap - RegionMap - !Ref 'AWS::Region’ - AMI ```

Use Cases:

Automating selection of region-specific resources (e.g., AMIs, endpoints).

Transforms

The `Transform` section specifies macros that CloudFormation uses to perform custom processing on parts of a template. This can include using `AWS::Serverless` for the Serverless Application Model (SAM) or `AWS::Include` to embed external code snippets from S3.

supported_transforms: ['AWS::Serverless', 'AWS::Include']

example_json: ```json Resources: MyBucket: Type: AWS:S3::Bucket Properties: Fn:: Transform: - Name: AWS::Include Parameters: Location: s3://bucket/myBucketName.json - Name: AWS::Include Parameters: Location: s3://bucket/myBucketAcl.json ```

Use Cases:

Simplifying serverless application deployments
Modularizing templates with external snippets.

Conditions

Conditions define when a resource is created or a property is defined. They allow for conditional logic within the template, such as creating resources only for a 'prod' environment type using `Fn::Equals` and `Ref` pseudo-parameters. **Example:** ```json Parameters: EnvType: Description: Environment type. Default: test Type: String AllowedValues: - prod - test ConstraintDescription: must specify prod or test. Conditions: CreateProdResources: Fn::Equals: - Ref: EnvType - prod Resources: EC2Instance: Type: AWS::EC2::Instance Properties: Imageld: Fn::FindInMap: - RegionMap - Ref: AWS::Region - AMI MountPoint: Type: AWS::EC2::VolumeAttachment Condition: CreateProdResources Properties: Instanceld: Ref: EC2Instance Volumeld: Ref: NewVolume Device: "/dev/sdh" ```

Use Cases:

Deploying different resources based on environment (e.g., dev/prod)
Enabling/disabling features based on specific criteria.

Resources (Mandatory Section)

The `Resources` section is the only mandatory part of a CloudFormation template. It declares the AWS resources to be included in the stack, each with a logical identifier, a `Type` (identifying the resource type), and `Properties` appropriate for that resource type. The logical ID is used for internal template references, distinct from the physical ID assigned by AWS upon provisioning. **Example EC2 Instance (YAML):** ```yaml AWSTemplateFormatVersion: '2010-09-09' Resources: MyEC2Instance: Type: AWS::EC2::Instance Properties: ImageId: ami-0c55b159cbfafe1f0 # Linux AMI InstanceType: t2.micro ``` **Example S3 Bucket (YAML):** ```yaml Resources: MyS3Bucket: Type: AWS::S3::Bucket Properties: BucketName: HelloWorldWebsite AccessControl: PublicRead WebsiteConfiguration: IndexDocument: index.html ``` **Example Security Group (YAML):** ```yaml Resources: MySecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Enable SSH access via port 22 SecurityGroupIngress: - IpProtocol: tcp FromPort: '22' ToPort: '22' CidrIp: 0.0.0.0/0 ```

Use Cases:

Defining all infrastructure components of a stack.

Outputs

The `Outputs` section declares output values that can be imported to other stacks (creating cross-stack references), returned to describe stack calls, or displayed on the console. Outputs provide a way to retrieve important information about the created resources. **Example (JSON):** ```json Outputs: BackupLoadBalancer DNSName: Description: The DNSName of the backup load balancer Value: Fn::GetAtt: - BackupLoadBalancer - DNSName InstancelD: Description: The Instance ID Value: Ref: EC2Instance ```

Use Cases:

Facilitating communication between different CloudFormation stacks
Displaying key resource identifiers (e.g., DNS names, IDs) after deployment.

CloudFormation Intrinsic Functions

Intrinsic functions are built-in functions in CloudFormation that help you manage your stacks by performing operations like concatenating strings, retrieving resource attributes, or conditionally creating resources.

These functions are processed by CloudFormation before the stack is created or updated, allowing for dynamic and flexible template definitions.

Fn::Base64

Returns the Base64 representation of a string.

Use Cases:

Encoding UserData for EC2 instances.

Fn::Cidr

Returns an array of CIDR address blocks. The number of CIDR blocks depends on the `count` parameter.

json_syntax: { "En::Cidr": [ipBlock, count, cidrBits]}

yaml_syntax: Fn.:Cidr - ipBlock - count - cidrBits

yaml_short_form: !Cidr [ ipBlock, count, cidrBits ]

ip_block: The user-specified CIDR address block to be split into smaller CIDR blocks.

count: The number of CIDRs to generate. Valid range is between 1 and 256.

cidr_bits: The number of subnet bits for the CIDR. E.g., '8' for a mask of '/24'.

Use Cases:

Dynamically creating subnet CIDR blocks.

Fn::FindInMap

Returns the value corresponding to keys in a two-level map that is declared in the Mappings section.

json_syntax: {"En::FindInMap" : [ "MapName", "TopLevelKey", "SecondLevelKey"] }

yaml_syntax: Fn::FindInMap: [ MapName, TopLevelKey, SecondLevelKey ]

yaml_short_form: !FindInMap [ MapName, TopLevelKey, SecondLevelKey ]

Use Cases:

Retrieving region-specific AMI IDs, or other configuration values from mappings.

Fn::GetAtt

Returns the value of an attribute from a resource in the template.

json_syntax: {"En::GetAtt" : [ "logicalNameOfResource", "attributeName" ]}

yaml_syntax: Fn::GetAtt: [ logicalNameOfResource, attributeName ]

yaml_short_form: !GetAtt logicalNameOfResource.attributeName

example_ec2_public_dns_json: ```json {"Fn::GetAtt" : [ "WebServerInstance", "PublicDnsName" ]} ```

example_ec2_public_dns_yaml: ```yaml Fn::GetAtt: [ WebServerName, PublicDnsName ] or !GetAtt WebServerName.PublicDnsName ```

Use Cases:

Retrieving dynamically generated resource attributes (e.g., DNS names, ARN, IDs).

Fn::GetAZs

Returns an array that lists Availability Zones for a specified region. This function enables template authors to write templates that adapt to the calling user's access, as customers have access to different Availability Zones. Specifying an empty string is equivalent to specifying `AWS::Region`.

json_syntax: {"Fn::GetAZs": "region" }

yaml_syntax: Fn::GetAZs: region

yaml_short_form: !GetAZs region

valid_inputs_us_east_1: { "Fn::GetAZs" : ''' }

valid_inputs_aws_region: {"Fn::GetAZs" : {"Ref" : "AWS::Region" } }

valid_inputs_explicit_region: {"Fn::GetAZs" : "us-east-1" }

Use Cases:

Dynamically deploying resources across available Availability Zones.

Fn::ImportValue

Returns the value of an output exported by another stack. This function is typically used to create cross-stack references.

json_syntax: {"En:ImportValue" : sharedValueToImport }

yaml_syntax: Fn::ImportValue: sharedValueToImport

yaml_short_form: !ImportValue sharedValueTolmport

example_json: ```json { "Fn:ImportValue" : {"Fn::Sub":"${NetworkStack}-SubnetID" } } ```

example_yaml: ```yaml Fn::ImportValue: !Sub "${NetworkStack}-SubnetID" ```

Use Cases:

Referencing resources created in a separate infrastructure stack (e.g., VPC, Subnet IDs)

Fn::Join

Appends a set of values into a single value, separated by the specified delimiter. If the delimiter is an empty string, the values are concatenated with no delimiter.

json_syntax: {"Fn:Join" : [ "delimiter", [ comma-delimited list of values ]]}

yaml_syntax: Fn:Join: [ delimiter, [ comma-delimited list of values ]]

yaml_short_form: !Join [ delimiter, [ comma-delimited list of values ]]

example_json: ```json "Fn:Join" : ["-", [ "CloudFormation", "Deep", "Dive" ]] ```

example_yaml: ```yaml Fn::Join: [-, [ CloudFormation, Deep, Dive ]] or Join [- [ CloudFormation, Deep, Dive 1] ```

Use Cases:

Constructing strings like ARN values, file paths, or resource names.

Fn::Length

Returns the number of elements in an array or the number of characters in a string.

Use Cases:

Determining the size of a list or string length for conditional logic.

Fn::Select

Returns a single object from a list of objects by index. The index must be a value from zero to N-1, where N is the number of elements in the list.

json_syntax: { "F::Select" : [ index, listOfObjects ]}

yaml_syntax: Fn::Select: [ index, listOfObjects ]

yaml_short_form: !Select [ index, listOfObjects ]

example_json: ```json { "Fn::Select": ["2", [ "VPC", "IGW", "Route", "Key" ]] } ```

example_yaml: ```yaml En::Select: [2, "VPC", "IGW", "Route", "Key" ] or !Select ["2", [ "VPC", "IGW", "Route", "Key" ]] ```

Use Cases:

Extracting specific values from an array, such as a particular subnet from a list of subnets.

Fn::Split

Splits a string into a list of string values using a specified delimiter. The `Fn::Select` function can then be used to pick a specific element from the resulting list.

json_syntax: { "Fn::Split" : [ "delimiter", "source string" ]}

yaml_syntax: Fn::Split: [ delimiter, source string ]

yaml_short_form: !Split [delimiter, source string ]

example_splitting_imported_value_json: ```json { "Fn:Select" : ["2", {"Fn::Split":['.',{"Fn:ImportValue": "AccountSubnetIDs"}]}] } ```

Use Cases:

Parsing values from strings (e.g., extracting components from an ARN).

Fn::Sub

Substitutes variables in an input string with values that you specify. This function can be used to construct command-line strings or outputs that include values only available after stack creation or update.

json_syntax: {"Fn::Sub" : [ String, { Var1Name: Var1Value, Var2Name: Var2Value } ] }

yaml_syntax: Fn::Sub: - String - {Var1 Name: Var1Value, Var2Name: Var2Value }

yaml_short_form: !Sub - String - { Var1Name: Var1Value, VarName: Var2Value }

Use Cases:

Creating dynamic strings based on runtime values or resource attributes.

Fn::ToJsonString

Converts an object into a JSON string.

Use Cases:

Passing complex objects as string values, for instance, in UserData.

Fn::Transform (Macro)

Specifies a macro to perform custom processing on part of a stack template.

json_syntax: {"Fn::Transform" : { "Name" : macro name, "Parameters" : {key: value,}}}

yaml_syntax: Fn::Transform Name: macro name Parameters: Key : value

yaml_short_form: !Transform { "Name" : macro name, "Parameters" : {key : value, .. } }

Use Cases:

Applying Serverless Application Model (SAM) transformations or including external snippets.

Ref

Returns the value of the specified parameter or resource. If you specify a parameter's logical name, it returns the value of the parameter. If you specify a resource's logical name, it returns a value typically used to refer to that resource, such as a physical ID.

json_syntax: {"Ref" : "logicalName" }

yaml_syntax: Ref: logicalName

yaml_short_form: !Ref logicalName

Use Cases:

Referencing parameter values or resource logical IDs within a template.

CloudFormation Condition Functions

CloudFormation provides condition functions to conditionally create stack resources or define properties based on logical evaluations.

The following functions are used to conditionally create stack resources: * `And` * `Equals` * `If` * `Not` * `Or`

CloudFormation Pseudo Parameters

Pseudo parameters are predefined by CloudFormation, acting similarly to environment variables, and can be reliably referenced within templates.

These parameters are automatically set and maintained by CloudFormation, ensuring consistency and correctness. They are referenced using the `Ref` Intrinsic Function.

* `AWS::AccountId`: Returns the AWS account ID of the account. * `AWS::NotificationARNs`: Returns the list of notification ARNs for the current stack. * `AWS::StackId`: Returns the ID of the stack. * `AWS::StackName`: Returns the name of the stack. * `AWS::Region`: Returns a string representing the AWS Region in which the resource is being created. **Example: Setting a Tag with Region using !Join and !Ref** ```yaml Tags: - Key: "Name" Value: !Join - "" - - "EC2 Instance for " - !Ref AWS::Region ```

CloudFormation Tools and Helper Scripts

CloudFormation provides tools like Designer for visual template creation and helper scripts for instance bootstrapping to enhance infrastructure management.

CloudFormation Designer is a visual tool that provides a drag-and-drop interface for adding resources to templates. It supports both JSON and YAML formats.

The `UserData` property for an EC2 instance allows scripts to be executed on the instance during its first boot cycle. This data is Base64 encoded and works for both Linux and Windows. It's important to be aware of the time it takes for these scripts to execute.

While procedural scripting is often not ideal, CloudFormation provides Python-based helper scripts to optimize instance bootstrapping. These scripts come preinstalled on Amazon Linux but are not executed automatically. They are updated periodically and can be updated using `yum install -y aws-cfn-bootstrap`. **Available Helper Scripts:** * `cfn-init`: Reads and interprets Metadata to execute `AWS::CloudFormation::Init`. * `cfn-signal`: Used to signal when a resource or application is ready. * `cfn-get-metadata`: Used to retrieve metadata based on a specific key. * `cfn-hup`: Used to check for updates to metadata and execute custom hooks.

CloudFormation in Operational Excellence & Disaster Recovery

CloudFormation plays a key role in achieving operational excellence through automation and is a vital component in disaster recovery strategies.

Within the AWS Well-Architected Framework, 'Operational Excellence' emphasizes performing operations as code. CloudFormation is explicitly listed as a tool for automating deployments using infrastructure as code (IaC) templates. It is also given as an example for automated deployments. (source_page: 3, 9)

CloudFormation can be a vital part of disaster recovery, though the specific mechanisms are covered in more advanced topics, such as the course wrap-up in a deep-dive course. (source_page: 4)

Learning Objectives

Infrastructure as Code (IaC)

CloudFormation Fundamentals - Terminology

CloudFormation Template Anatomy - Top-Level Sections

AWSTemplateFormatVersion

Description

Metadata

Parameters

Mappings

Conditions

Transform

Resources

Outputs

CloudFormation Template Languages - JSON and YAML

CloudFormation Template Sections - Advanced Configuration

Parameters

Mappings

Transforms

Conditions

Resources (Mandatory Section)

Outputs

CloudFormation Intrinsic Functions

Fn::Base64

Fn::Cidr

Fn::FindInMap

Fn::GetAtt

Fn::GetAZs

Fn::ImportValue

Fn::Join

Fn::Length

Fn::Select

Fn::Split

Fn::Sub

Fn::ToJsonString

Fn::Transform (Macro)

Ref

CloudFormation Condition Functions

CloudFormation Pseudo Parameters

CloudFormation Tools and Helper Scripts

CloudFormation in Operational Excellence & Disaster Recovery

Exam Tips

Glossary

Key Takeaways

Content Sources