Opportunity Name: Kinesis Optimize Logging
AWS Resource Type: Amazon Kinesis Data Streams
Opportunity Description:
This Finder identifies Kinesis Data Streams that are incurring unnecessary costs due to excessive or redundant logging configurations — specifically, logging every read and write action through CloudWatch. While these logs can be valuable for debugging and auditing, they often remain enabled in production environments where their utility diminishes and costs accumulate rapidly.
CloudFix highlights opportunities to reduce these costs by modifying or disabling logging configurations where appropriate, helping you streamline observability expenses without compromising essential monitoring.
Criteria for identifying the opportunity:
- The Kinesis stream has Enhanced Monitoring enabled for all shard-level metrics.
- The CloudWatch logging configuration includes PutRecord, PutRecords, GetRecords, GetShardIterator metrics at high granularity (e.g., per second/per shard).
- Logs are being pushed continuously to CloudWatch but show limited diagnostic utility (low volume of alerts, no downstream consumers using them).
- The stream operates in a stable production environment with no recent troubleshooting activity or major incident history.
Potential Savings (if known):
While savings vary by volume and region, CloudWatch logs for high-throughput Kinesis streams can cost hundreds to thousands of dollars annually per stream. Disabling unnecessary shard-level logs can cut those costs significantly — often by 30-70%, depending on the size and activity level of the stream.
What happens when the Fixer is executed?
This Finder does not currently have an automatic Fixer. Customers must manually modify logging configurations using the AWS Console or AWS CLI. The process involves:
- Navigating to the Kinesis stream in the AWS Console.
- Selecting “Enhanced Monitoring.”
- Disabling specific shard-level metrics that are not critical for ongoing monitoring.
CloudFix provides recommendations for which metrics to disable and why, along with guidance for maintaining essential monitoring coverage.
Is it possible to roll back once CloudFix implements the fixer?
Yes. Monitoring settings can be re-enabled at any time with no disruption to stream operations. This change is fully reversible.
Can CloudFix implement the fix automatically once I accept the recommendation?
No. Customers need to apply the change manually or engage CloudFix Professional Services for implementation.
Does this fix require downtime?
No. Adjusting Kinesis monitoring settings is a live operation that does not affect stream performance or data ingestion.
Additional Resources:
Bill Gleeson
Comments