CloudWatch - AWS Study Guide (v2.0)

Overview of Amazon CloudWatch

Amazon CloudWatch serves as the central monitoring and observability service within AWS, designed to provide comprehensive insights into the health and performance of AWS resources and applications.

Amazon CloudWatch is a monitoring and management service for AWS resources and the applications you run on AWS. Its primary function is to track the performance and health of your resources, allowing you to react to potential problems before they impact your users. You can collect and monitor log files from various sources, including Amazon EC2 instances, AWS CloudTrail, Amazon Route 53, and many others. CloudWatch is a distributed statistics-gathering system that collects and tracks metrics from your applications. You can also create and use your own custom metrics and receive notifications when an alarm is triggered.

CloudWatch is a monitoring and observability platform designed to give insight into AWS architecture and identify potential issues before they become bigger problems. Monitoring is necessary to catch warning signs before catastrophe strikes. It can collect system-level and application-level metrics to identify problems with the service, operating system, or resources.

While Amazon CloudWatch inherently offers robust capabilities for monitoring, including logs, metrics, and dashboarding, it doesn’t capture all relevant data points by default. The CloudWatch Agent is crucial for addressing these gaps, enabling the collection of metrics like memory and disk space utilization, as well as monitoring custom log file locations and creating custom metrics with unique namespaces.

CloudWatch comprises three main components: Metrics, Alarms, and Events.

CloudWatch offers two monitoring options for Amazon EC2 instances:

Provides seven pre-selected metrics at a 5-minute frequency and three status check metrics at a 1-minute frequency, at no extra charge.

Provides all metrics available in Basic Monitoring at a 1-minute frequency, for an additional charge. Offers data aggregation by Amazon EC2, Amazon Machine Image (AMI) ID, and instance type.

CloudWatch can help detect suspicious activity such as unusual spikes in service usage (CPU, disk, Amazon RDS) or anomalous billing metrics (requires enabling this feature in account settings). This can be crucial for detecting unauthorized access or compromised credentials.

CloudWatch is an example of a tool used for monitoring within the Operational Excellence pillar, which focuses on running and monitoring systems efficiently.

CloudWatch is used to enable traceability by logging all activities in the application landscape, helping to track resource creation/deletion and API access, supporting the Security pillar which focuses on protecting data and systems.

CloudWatch Metrics

Metrics are fundamental to CloudWatch, representing time-ordered sets of data points that quantify the performance and health of your resources and applications.

Metrics are time-ordered sets of data points representing the values of a monitored variable over time. Examples include CPU utilization, network traffic, and disk I/O.

A container for related metrics. AWS namespaces follow the convention AWS/<service> (e.g., AWS/EC2). User-defined metrics can be grouped by user-defined namespaces.

Technical Specs: AWS namespaces convention: AWS/<service>

Name-value pairs that further categorize metrics (e.g., InstanceId for CPU utilization). Adding dimensions allows for more granular monitoring and analysis.

The time interval over which metrics are collected (in seconds). Shorter periods provide more frequent updates but increase data volume.

There are two kinds of metrics in CloudWatch: default metrics that come with every service and custom metrics that require the CloudWatch agent to be installed in the operating system.

Custom metrics are user-defined metrics, grouped by user-defined namespaces, published using the AWS CLI, an API, or a CloudWatch agent.

Standard metrics have a reporting interval of 5 minutes.

Technical Specs: Reporting interval: 5 minutes

Detailed metrics have a reporting interval of 1 minute and an additional cost.

Technical Specs: Reporting interval: 1 minute; Additional cost: Yes

CloudWatch Alarms

CloudWatch Alarms enable you to monitor metrics and automatically trigger actions when predefined thresholds are breached, ensuring proactive response to operational issues.

CloudWatch alarms have three states:

The metric is within the defined threshold.

The metric is outside the defined threshold. Important: An ALARM state doesn’t automatically signify an emergency; it simply indicates that the metric has exceeded (or fallen below) the threshold.

The alarm is new, the metric is unavailable, or there isn’t enough data to determine the state.

You can create CloudWatch alarms that monitor single metrics or mathematical expressions based on multiple metrics. Alarms trigger actions when a metric crosses a defined threshold for a specified duration. The action can be anything from sending an email to automatically scaling resources.

When the threshold is breached, you can choose to receive a notification by email or SMS, or you can choose to take an automated action such as stopping an EC2 instance or triggering an auto-scaling group.

Composite alarms can be used to combine alarms (e.g. summarized, aggregated health indicator).

Monitors estimated AWS charges and generates alerts when charges exceed a specified threshold. Billing metric data is stored in the US East (Northern Virginia) Region, representing worldwide charges. Sends alarm notifications via email through an Amazon SNS topic.

CloudWatch Events

Amazon CloudWatch Events is a serverless event bus that provides a near real-time stream of system events, enabling automated responses to changes in your AWS resources.

CloudWatch Events provides a continuous stream of system events reflecting changes in AWS resources. You can configure simple rules to match events and direct them to various target functions or streams (e.g., AWS Lambda, Amazon SNS, Amazon SQS). It enables automated actions, creating custom rules and targets for event actions, and integration with other AWS services. This results in automation of tasks, improved operational efficiency, reduced manual intervention, and enhanced security and compliance.

Events indicate changes in the AWS environment (e.g., an EC2 instance changing state). Custom application-level events can also be published.

Targets process events; examples include EC2 instances, AWS Lambda functions, Amazon SNS topics, and Amazon SQS queues.

Rules match incoming events and route them to targets for processing. A single rule can have multiple parallel targets.

CloudWatch Events can be used to schedule automated actions that self trigger at specific times using cron or rate expressions. For example, each time Amazon EC2 Auto Scaling launches an EC2 instance, a specified AWS Systems Manager Run Command script can be automatically executed using a CloudWatch Events rule. Another example is creating a real-time notification that informs you when an instance is stopped or terminated.

CloudWatch Agent

The CloudWatch Agent is a software package designed to automatically collect system-level and application-specific metrics and logs from your IT infrastructure, extending Amazon CloudWatch's monitoring capabilities beyond default metrics.

The CloudWatch Agent is a software daemon that runs continuously on your servers, proactively collecting specified metrics and logs and sending them to Amazon CloudWatch for analysis and visualization.

Purpose and Role

The CloudWatch Agent is a software package designed to automatically collect system-level and application-specific metrics and logs from your IT infrastructure. Its primary purpose is to enhance the monitoring capabilities of Amazon CloudWatch, extending its reach beyond the default metrics provided.

Definition

Supported Installation Environments

The agent can be installed on a variety of platforms.

amazon_ec2_instances: Supporting Linux, Windows, and macOS

hybrid_environments: Including virtual machines on Azure, Google Cloud Platform (GCP)

on_premise_workloads: Servers running in your own data centers

Key Use Cases - Monitoring Resource Utilization (Memory, Disk)

CloudWatch does not track RAM usage or available disk space by default. The agent collects these vital metrics, providing essential visibility into resource constraints.

Key Use Cases - Custom Log File Monitoring

For applications that write logs to specific files (e.g., web server access and error logs), the agent can be configured to stream these logs to CloudWatch Logs for analysis, alerting, and archival.

Key Use Cases - Custom Metrics and Namespaces

Beyond standard system metrics, the agent allows you to define and send custom metrics with unique namespaces, categorizing your application-specific performance data.

Key Use Cases - Advanced System and Application Metrics

It enables the collection of detailed operating system-level metrics or application-specific performance data that are not exposed through default CloudWatch mechanisms.

Key Use Cases - Containerized Environments

The agent is also valuable for collecting logs and metrics from containerized applications and microservices, providing insights into their operational health.

Prerequisites - Software Installation

The CloudWatch Agent software must be installed on the target machine.

Prerequisites - Permissions and Authentication (EC2 Instances)

The recommended approach is to assign an Instance Role to the EC2 instance. This role should have policies attached that grant the necessary permissions for the agent to communicate with CloudWatch services (e.g., CloudWatchAgentAdminPolicy).

Prerequisites - Permissions and Authentication (On-Premise/Other Cloud Providers)

For workloads not running on EC2, you will need to create IAM Access Key and Secret Access Key credentials and configure the agent to use them.

Prerequisites - Network Configuration (Security Groups)

Outbound network access is necessary for the agent to communicate with CloudWatch. For EC2 Instances, ensure that the instance’s Security Group has outbound rules configured to allow traffic to the necessary CloudWatch endpoints. While default security groups usually permit all outbound traffic, custom security groups require explicit configuration.

CloudWatch Agent Installation and Configuration on Linux EC2

procedure

This section outlines the steps for installing and configuring the CloudWatch Agent on a Linux EC2 instance to collect custom metrics and logs.

A detailed demonstration of the end-to-end process for installing, configuring, and verifying the CloudWatch Agent on a Linux EC2 instance.

Prerequisites

Linux-based EC2 instance ready in AWS Management Console
IAM Role with CloudWatchAgentAdminPolicy and AmazonSSMFullAccess policies
Security Group allowing outbound traffic to CloudWatch endpoints

Initial CloudWatch Metrics Review (Default)

💡 To observe default metrics (CPU utilization, disk I/O, network traffic) and note the absence of memory usage and disk space availability.

Create and Assign IAM Role

💡 The agent requires appropriate permissions to send data to CloudWatch. For EC2 instances, assigning an Instance Role is the recommended approach.

Create Role in IAM dashboard: Select 'AWS service' as trusted entity and 'EC2' as service.

Attach Policies: Attach 'CloudWatchAgentAdminPolicy' and 'AmazonSSMFullAccess'.

Name Role: Provide a descriptive name (e.g., ec2-instance-profile-for-demo).

Assign Role to EC2 instance: Go to EC2 instance, select 'Actions' > 'Security' > 'Modify IAM role,' and assign the newly created role.

Security Group Configuration

💡 Outbound network access is necessary for the agent to communicate with CloudWatch.

Create a new security group (e.g., demo-SecurityGroup).

Outbound Rules: Ensure the default outbound rule allowing all traffic is sufficient. Restricting this rule will disrupt agent communication.

Assign Security Group: Associate this security group with your EC2 instance.

System Preparation (HTTPD Server - Optional for log monitoring demo)

💡 To demonstrate custom log monitoring.

Install HTTPD: sudo yum install httpd -y

Create a sample index.html file.

Start and enable the service: sudo systemctl start httpd && sudo systemctl enable httpd.

Agent and CollectD Installation

Install the CloudWatch Agent: sudo yum install amazon-cloudwatch-agent -y

Install CollectD (for system stats): sudo amazon-linux-extras install collectd -y

Configuration Wizard Walkthrough

💡 The wizard generates a JSON configuration file and saves it to the Systems Manager Parameter Store.

Run the agent’s configuration wizard: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard.sh

Follow prompts to configure: OS, installation target (EC2), enable StatsD (optional), monitor metrics from CollectD, monitor host metrics (CPU, Memory), add EC2 dimensions (Instance ID, Type, Image ID), select metric collection resolution (basic or high resolution).

Configure log file monitoring (e.g., /var/log/httpd/access_log, /var/log/httpd/error_log) with custom log group and stream names.

Running the Agent with Configuration

💡 The agent will start, fetching the configuration from the specified SSM Parameter Store path.

Execute the agent: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:AmazonCloudWatch-linux -region <your-region>

CloudWatch Agent Installation and Configuration on Windows EC2

procedure

This section details the step-by-step installation and configuration of the Amazon CloudWatch agent on a Windows EC2 instance.

The process to install and configure the CloudWatch agent on a Windows EC2 instance, utilizing AWS Systems Manager for efficient deployment and management.

Prerequisites

Active AWS account
At least one Windows EC2 instance in a running state
CloudWatch Unified Agent software must be installed on the target instance
IAM instance role with AmazonCloudWatchAgentAdminPolicy, AmazonCloudWatchAgentServerPolicy, AmazonSSMFullAccess, and AmazonSSMManagedInstanceCore policies
Security group of the EC2 instance must allow outbound connections to AWS CloudWatch services

Creating an IAM Role for EC2 Instance

💡 The CloudWatch Agent requires appropriate permissions to communicate with AWS CloudWatch services.

Navigate: AWS Management Console -> IAM -> Roles -> Create role.

Service Selection: Choose 'EC2' as the service.

Permissions Policies: Attach 'AmazonCloudWatchAgentAdminPolicy', 'AmazonCloudWatchAgentServerPolicy', 'AmazonSSMFullAccess', and 'AmazonSSMManagedInstanceCore'.

Role Naming: Provide a descriptive name (e.g., CES Cloud Expert Solution IAM).

Assignment: Attach the newly created IAM role to the running Windows EC2 instance via the EC2 console (Security tab -> Modify IAM role).

Installing CloudWatch Agent using AWS Systems Manager (SSM)

💡 SSM Run Command provides a scalable solution for deploying and managing agents across multiple instances.

Verify Instance Readiness: Ensure the EC2 instance appears as a managed instance in the Systems Manager console (Fleet Manager). This can take 30 seconds to over a minute.

Navigate: Systems Manager -> Run Command.

Command Document: Select 'AWS-ConfigureAWSPackage'.

Action: Select 'Install'.

Installation Type: Select 'Uninstall and reinstall' (to ensure a clean install).

Package Name: Enter 'AmazonCloudWatchAgent'.

Version: Enter 'latest'.

Target Selection: Choose 'Manual' and select the single Windows EC2 instance.

Execute: Click 'Run'. The status will change to 'Success' upon successful installation.

Configuring the CloudWatch Agent

💡 The configuration wizard helps set up metrics and log collection, and stores the configuration in SSM Parameter Store.

Access: Remote Desktop Protocol (RDP) into the Windows EC2 instance.

Verification of Installation: Navigate to C:\Program Files\Amazon\Amazon CloudWatch Agent\ to confirm the agent’s presence.

Launch Configuration Wizard: Open Command Prompt as Administrator, navigate to cd "C:\Program Files\Amazon\Amazon CloudWatch Agent\", and run AmazonCloudWatchAgentConfigWizard.exe.

Follow wizard prompts: Select Windows OS, use EC2 instance metadata for region, identify as EC2 instance, enable StatsD (yes), accept default port, collection interval 10 seconds, aggregation interval 60 seconds. Do not import existing configuration.

Configure Host Metrics: Select 'yes' to monitor CPU, Memory, 'yes' for CPU per core, 'yes' for EC2 dimensions, 'yes' to aggregate EC2 dimensions. Accept default 60 seconds for High Resolution Metrics.

Configure Custom Log File Monitoring: Select 'yes' if needed, provide path (e.g., var/log/app.log), choose default log stream name (Instance ID), specify log retention (e.g., 5 days). Select 'no' for additional log files. Select 'no' for Windows Event Log Monitoring and X-Ray Traces if not needed.

Store Config in SSM Parameter Store: Select 'yes'. Use default parameter store name 'AmazonCloudWatchWindows' or provide custom. Confirm region and use default credentials. A success message will appear: 'Successfully put config to parameter store. Press enter to exit.'

Starting CloudWatch Agent Service

💡 To activate the agent and begin collecting metrics and logs.

Navigate: Systems Manager -> Run Command.

Command Document: Select 'AmazonCloudWatch-ManageAgent'.

Action: Select 'Configure'.

Mode: Select 'EC2'.

Optional Configuration Location: Enter the parameter store name (e.g., 'AmazonCloudWatchWindows').

Optional Restart: Select 'Yes'.

Target Selection: Manual selection of the EC2 instance. Click 'Run'.

CloudWatch Logs

CloudWatch Logs is a fundamental AWS service for centralized log management, enabling monitoring, storage, and access to log files from various sources across your AWS environment.

CloudWatch Logs is a tool provided by AWS to help monitor, store, and access log files from various sources. It centralizes the logs from all of your systems, applications, and AWS services that you use, in a single, highly scalable service. It enables efficient monitoring, storage, and access of log files from various sources.

Logging events are individual data points that contain the data and a timestamp of when that data occurred.

Log streams are collections of logging events from a single source, such as a single EC2 instance serving a website.

Logging groups group together multiple log streams from different sources, such as all the web server logs from multiple EC2 instances. Each log group represents a specific log type with a defined format.

The typical log analysis process comprises three phases: Configure, Collect, and Analyze.

Determine the information to save, file format, delimiters, and fields for each service.

Upload log files from various cloud resources (e.g., Amazon S3, Amazon EMR) to a central location. CloudWatch Logs uses the CloudWatch agent for automatic log collection from supported services (e.g., EC2 instances).

Use analytics tools (like CloudWatch Logs Insights) to examine data, identify errors, investigate issues, monitor releases, and track trends.

Metric filters enable searching for specific string patterns within log groups. Each match increments a custom CloudWatch metric, which can trigger alarms or notifications. Filter Patterns define filter patterns using a specific syntax. Patterns are case-sensitive. Multiple terms are allowed, but all must be present in the log event for a match.

CloudWatch Logs Insights provides a query language for querying logs and creating visualizations. It allows users to query all logs using SQL-like commands to look for general trends.

You can configure a CloudWatch Logs log group to stream data it receives to your Amazon OpenSearch Service cluster in near real-time through a CloudWatch Logs subscription.

CloudWatch Logs can be used with various log-creating services, including EC2, Lambda, RDS, CloudTrail, and on-premises resources.

CloudWatch Logs is agent-based, meaning that it requires an agent to be installed on the host in order to stream logs to CloudWatch Logs. The CloudWatch agent can be used for both streaming logs and shipping custom metrics. Once the agent is installed, a configuration file can be generated using the config wizard. The configuration file is a JSON document that specifies the logs to be monitored and their location. After the configuration file is generated, the agent can be loaded with the configuration file and started to begin streaming logs to CloudWatch Logs.

Creating a CloudWatch Logs Metric Filter and Alarm

procedure

This procedure outlines how to create a metric filter in CloudWatch Logs to identify specific patterns (e.g., 404 errors) and then set up an alarm based on that metric.

Configure a filter to identify 404 Errors in a log file and then set up an alarm to send a notification when too many 404 Not Found errors are received.

Prerequisites

EC2 instance with web server generating access logs (e.g., Apache HTTPD)
CloudWatch Agent configured to stream web server access logs to a CloudWatch Log Group (e.g., HttpAccessLog)
An email address to receive SNS notifications

Generate Log Data

💡 To populate the log group with data that includes the pattern you intend to filter (e.g., 404 errors).

Access the web server's public IP in a browser to generate access logs.

Attempt to access non-existent pages (e.g., append /start to the URL) multiple times to generate 404 errors in the access logs.

Navigate to CloudWatch Log Groups

From the Services menu, choose CloudWatch.

In the left navigation pane, choose Log groups. Verify 'HttpAccessLog' is listed.

Create a Metric Filter

💡 To identify specific log patterns (e.g., 404 errors) from the log data.

Select the check box next to 'HttpAccessLog'.

From the Actions dropdown menu, select 'Create metric filter'.

Paste the Filter pattern: '[ip, id, user, timestamp, request, status_code=404, size]'

💡 This pattern tells CloudWatch Logs how to interpret fields and filters for lines with 'status_code=404'.

[ip, id, user, timestamp, request, status_code=404, size]

In the Test pattern section, select the EC2 instance ID from the dropdown.

Click 'Test pattern' and then 'Show test results' to verify that at least one result with a $status_code of 404 is found.

Click 'Next'.

In the Create filter name section, enter '404Errors' as the Filter name.

In the Metric details section, configure: Metric namespace: 'LogMetrics', Metric name: '404Errors', Metric value: '1'.

Click 'Next' and then 'Create metric filter'.

Create an Alarm using the Filter

💡 To send a notification when the number of 404 errors exceeds a predefined threshold.

In the '404Errors' panel (metric filter), choose the check box in the top-right corner.

In the Metric filters section, choose 'Create alarm'.

Configure settings: Period: '1 minute'. Condition: 'Whenever 404Errors is: Greater/Equal than: 5'.

Click 'Next'.

In the Notification section, select 'Create new topic'. Enter an email address that you can access. Click 'Create topic'.

Click 'Next'.

For Name and description, configure: Alarm name: '404 Errors', Alarm description: 'Alert when too many 404s detected on an instance'.

Click 'Next' and then 'Create alarm'.

Go to your email, look for a confirmation message, and select the 'Confirm subscription' link.

CloudWatch Integrations & Advanced Use Cases

CloudWatch extends its capabilities through seamless integration with numerous AWS services, enabling comprehensive monitoring, automation, and deeper insights across the AWS ecosystem.

AWS VPC Flow Logs captures information about the IP traffic going to and from network interfaces in your VPC. This data can be published to Amazon CloudWatch for monitoring, creating custom metrics, and alerts for potential security issues. CloudWatch Logs Insights can be used to query this data for trend analysis.

CloudWatch plays a critical role in Auto Scaling by monitoring EC2 instance metrics (CPU, memory, network, etc.). If a metric crosses a predefined threshold, CloudWatch triggers an alarm, which notifies the relevant Auto Scaling policy. This policy then instructs the Auto Scaling group to perform a scaling action (e.g., add or remove an instance).

Amazon RDS integrates seamlessly with CloudWatch, providing key performance metrics like CPU utilization, Disk Queue Depth, IO Latency, and IO Throughput. Performance Insights, a database performance monitoring feature, is also integrated with CloudWatch. Additionally, database logs can be exported to CloudWatch Logs for analysis.

AWS CloudTrail records API calls and actions, and securely delivers these logs to an S3 bucket. These logs can also be sent to Amazon CloudWatch Logs for real-time alerts on specific activities (e.g., critical security group modifications, IAM permission elevation attempts).

Amazon Managed Grafana is a fully managed AWS service for secure data visualization, querying, correlating, and visualizing operational metrics, logs, and traces from a variety of data sources. CloudWatch is one of the built-in data sources for Amazon Managed Grafana.

For on-premise or other cloud VMs managed by AWS Systems Manager (SSM), the non-admin IAM user and group for Systems Manager require 'CloudWatch read-only access' policy attached for monitoring purposes. This allows SSM to collect and display inventory data from managed instances, with monitoring insights available in CloudWatch.

Learning Objectives

Overview of Amazon CloudWatch

CloudWatch Metrics

CloudWatch Alarms

CloudWatch Events

CloudWatch Agent

Purpose and Role

Definition

Supported Installation Environments

Key Use Cases - Monitoring Resource Utilization (Memory, Disk)

Key Use Cases - Custom Log File Monitoring

Key Use Cases - Custom Metrics and Namespaces

Key Use Cases - Advanced System and Application Metrics

Key Use Cases - Containerized Environments

Prerequisites - Software Installation

Prerequisites - Permissions and Authentication (EC2 Instances)

Prerequisites - Permissions and Authentication (On-Premise/Other Cloud Providers)

Prerequisites - Network Configuration (Security Groups)

CloudWatch Agent Installation and Configuration on Linux EC2

Prerequisites

Initial CloudWatch Metrics Review (Default)

Create and Assign IAM Role

Create Role in IAM dashboard: Select 'AWS service' as trusted entity and 'EC2' as service.

Attach Policies: Attach 'CloudWatchAgentAdminPolicy' and 'AmazonSSMFullAccess'.

Name Role: Provide a descriptive name (e.g., ec2-instance-profile-for-demo).

Assign Role to EC2 instance: Go to EC2 instance, select 'Actions' > 'Security' > 'Modify IAM role,' and assign the newly created role.

Security Group Configuration

Create a new security group (e.g., demo-SecurityGroup).

Outbound Rules: Ensure the default outbound rule allowing all traffic is sufficient. Restricting this rule will disrupt agent communication.

Assign Security Group: Associate this security group with your EC2 instance.

System Preparation (HTTPD Server - Optional for log monitoring demo)

Install HTTPD: sudo yum install httpd -y

Create a sample index.html file.

Start and enable the service: sudo systemctl start httpd && sudo systemctl enable httpd.

Agent and CollectD Installation

Install the CloudWatch Agent: sudo yum install amazon-cloudwatch-agent -y

Install CollectD (for system stats): sudo amazon-linux-extras install collectd -y

Configuration Wizard Walkthrough

Run the agent’s configuration wizard: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard.sh

Follow prompts to configure: OS, installation target (EC2), enable StatsD (optional), monitor metrics from CollectD, monitor host metrics (CPU, Memory), add EC2 dimensions (Instance ID, Type, Image ID), select metric collection resolution (basic or high resolution).

Configure log file monitoring (e.g., /var/log/httpd/access_log, /var/log/httpd/error_log) with custom log group and stream names.

Running the Agent with Configuration

Execute the agent: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:AmazonCloudWatch-linux -region <your-region>

CloudWatch Agent Installation and Configuration on Windows EC2

Prerequisites

Creating an IAM Role for EC2 Instance

Navigate: AWS Management Console -> IAM -> Roles -> Create role.

Service Selection: Choose 'EC2' as the service.

Permissions Policies: Attach 'AmazonCloudWatchAgentAdminPolicy', 'AmazonCloudWatchAgentServerPolicy', 'AmazonSSMFullAccess', and 'AmazonSSMManagedInstanceCore'.

Role Naming: Provide a descriptive name (e.g., CES Cloud Expert Solution IAM).

Assignment: Attach the newly created IAM role to the running Windows EC2 instance via the EC2 console (Security tab -> Modify IAM role).

Installing CloudWatch Agent using AWS Systems Manager (SSM)

Verify Instance Readiness: Ensure the EC2 instance appears as a managed instance in the Systems Manager console (Fleet Manager). This can take 30 seconds to over a minute.

Navigate: Systems Manager -> Run Command.

Command Document: Select 'AWS-ConfigureAWSPackage'.

Action: Select 'Install'.

Installation Type: Select 'Uninstall and reinstall' (to ensure a clean install).

Package Name: Enter 'AmazonCloudWatchAgent'.

Version: Enter 'latest'.

Target Selection: Choose 'Manual' and select the single Windows EC2 instance.

Execute: Click 'Run'. The status will change to 'Success' upon successful installation.

Configuring the CloudWatch Agent

Access: Remote Desktop Protocol (RDP) into the Windows EC2 instance.

Verification of Installation: Navigate to C:\Program Files\Amazon\Amazon CloudWatch Agent\ to confirm the agent’s presence.

Launch Configuration Wizard: Open Command Prompt as Administrator, navigate to cd "C:\Program Files\Amazon\Amazon CloudWatch Agent\", and run AmazonCloudWatchAgentConfigWizard.exe.

Follow wizard prompts: Select Windows OS, use EC2 instance metadata for region, identify as EC2 instance, enable StatsD (yes), accept default port, collection interval 10 seconds, aggregation interval 60 seconds. Do not import existing configuration.

Configure Host Metrics: Select 'yes' to monitor CPU, Memory, 'yes' for CPU per core, 'yes' for EC2 dimensions, 'yes' to aggregate EC2 dimensions. Accept default 60 seconds for High Resolution Metrics.

Configure Custom Log File Monitoring: Select 'yes' if needed, provide path (e.g., var/log/app.log), choose default log stream name (Instance ID), specify log retention (e.g., 5 days). Select 'no' for additional log files. Select 'no' for Windows Event Log Monitoring and X-Ray Traces if not needed.

Store Config in SSM Parameter Store: Select 'yes'. Use default parameter store name 'AmazonCloudWatchWindows' or provide custom. Confirm region and use default credentials. A success message will appear: 'Successfully put config to parameter store. Press enter to exit.'

Starting CloudWatch Agent Service

Navigate: Systems Manager -> Run Command.

Command Document: Select 'AmazonCloudWatch-ManageAgent'.

Action: Select 'Configure'.

Mode: Select 'EC2'.

Optional Configuration Location: Enter the parameter store name (e.g., 'AmazonCloudWatchWindows').

Optional Restart: Select 'Yes'.

Target Selection: Manual selection of the EC2 instance. Click 'Run'.

CloudWatch Logs

Creating a CloudWatch Logs Metric Filter and Alarm

Prerequisites