CloudWatch Collect Slice
Overview
Amazon CloudWatch is a monitoring service for AWS cloud resources and the applications you run on AWS. You can use Amazon CloudWatch to collect and track metrics, collect and monitor log files, set alarms, and automatically react to changes in your AWS resources.
Datacoral’s CloudWatch slice is a collect (datasource) slice that collects data of CloudWatch metric statistics. With the slice, customers can conveniently separate out the different types of metrics and granularities into multiple tables in redshift to make them easy to analyze.
Steps to add this slice to your installation
The steps to launch your slice are:
- Generate AWS API keys
- Specify the slice config
- Add the CloudWatch slice
1. Generate AWS API keys
Setup requirements
Before getting started please make sure to have the following information:
- Access to an active AWS account
Setup instructions
- Generate a new AWS keys
a. Go to URL: https://console.aws.amazon.com/iam/home?region=us-east-1#/users
b. Select or create new user
c. On user details, open
Security Credentials
d. ClickCreate access key
button, and copy credentials
2. Specify the slice config
To get a template for the CloudWatch slice configuration save the output of the describe --input-parameters
command as follows:
Necessary input parameters:
accessKeyId
- your AWS Access Key IdsecretAccessKey
- your AWS Secret Keyregion
- your AWS region
Optional input parameters:
schedule
- in cron format
Loadunits:
Multiple loadunits can be specified in a single slice to pull different metrics and statistics into separate tables. Below is the sample format of the loadunits section of the input parameters file for the CloudWatch Metrics Collect Slice.
Note :
- Start time and End time needed to get statistics will be based on schedule of the load unit
- dimension_filter is a non-mandatory list of dimension_name, dimension values combination, if provided, then the data extracted will be only for the specified dimension values for that dimension. When not provided, data will be extracted for all dimension values, example:
- Period is in seconds - 5, 10, 30, 60, or any multiple of 60, default is 3600
Restrictions:
you can use up to 10 percentile statistics.
PREREQUISITES
n\a
INPUT PARAMETERS
To get a the starting template save the output of the describe --input-parameters
command as follows:
Necessary input parameters:
namespace
dimensions
for metrics
item:
name
statistics
Example template:
3. Add the Slice
slice-name
Name of your slice. A schema with your slice-name is automatically created in your warehouseparams-file
File path to your input parameters file. Ex. cloudwatch_parameters_file.json
Supported load units
cloudwatch
Slice output
Output of this slice is stored in S3 and Redshift.
AWS S3
Data stored in AWS S3 is partitioned by date and time in the following bucket
s3://datacoral-data-bucket/<sliceName>
AWS Redshift: Schema - schema name will be same as a slice-name. Tables produced by the slice are:
Questions? Interested?
If you have questions or feedback, feel free to reach out at hello@datacoral.co or Request a demo