AWS Streaming Data Solution for Amazon MSK

AWS Streaming Data Solution for Amazon MSK

ยท

3 min read

โžก๏ธ Option 1: AWS CloudFormation template using Amazon Managed Streaming for Apache Kafka (Amazon MSK). Deploy a solution that automatically configures the AWS services necessary to easily capture, store, process, and deliver streaming data using Amazon MSK. To deploy this solution using the available AWS CloudFormation template, select Deploy with AWS.

2022-07-30 (27).png

1) An Amazon MSK cluster.

2) An Amazon EC2 instance that contains the Apache Kafka client libraries required to communicate with the Amazon MSK cluster. This client machine is located on the same VPC as the cluster, and it can be accessed via AWS Systems Manager Session Manager.

More Details here: ๐Ÿ‘‰ Link

โžก๏ธ Option 2: AWS CloudFormation template using Amazon MSK and AWS Lambda Deploy a solution that automatically configures the AWS services necessary to easily capture, store, process, and deliver streaming data using Amazon MSK. To deploy this solution using the available AWS CloudFormation template, select Deploy with AWS.

2022-07-30 (28).png

1) An AWS Lambda function that processes process records in a Kafka topic. The default function is a Node.js application that logs the received messages, but it can be customized to meet your business needs.

More Details here: ๐Ÿ‘‰ Link

โžก๏ธ Option 3: AWS CloudFormation template using Amazon MSK, AWS Lambda, and Amazon Kinesis Firehose. Deploy a solution that automatically configures the AWS services necessary to easily capture, store, process, and deliver streaming data using Amazon MSK. To deploy this solution using the available AWS CloudFormation template, select Deploy with AWS.

2022-07-30 (29).png

1) An AWS Lambda function that processes process records in a Kafka topic.

2) An Amazon Kinesis Data Firehose delivery stream that buffers data before delivering it to the destination. 3) An Amazon S3 bucket that stores all original events from the Amazon MSK cluster.

More Details here: ๐Ÿ‘‰

Link

Option 4: AWS CloudFormation template using Amazon MSK, Amazon Kinesis Data Analytics, and Amazon S3. Deploy a solution that automatically configures the AWS services necessary to easily capture, store, process, and deliver streaming data using Amazon MSK. To deploy this solution using the available AWS CloudFormation template, select Deploy with AWS.

2022-07-30 (30).png

1) An Amazon Kinesis Data Analytics application that reads events from an existing topic in an Amazon MSK cluster.

2) An Amazon S3 bucket that stores the output of the demo application.

More Details here: ๐Ÿ‘‰

Link

You can download the Implementation Guide from this link ๐Ÿ‘€๐Ÿ‘‡ Link

I hope you will find it useful for your organization! ๐Ÿค“

[IMPORTANT]

  • This material is not mine. This material was written exclusively by Amazon Web Services.

I am sharing this with a bigger IT audience to help folks in their quest to learn AWS. I don't intend to monetize anything, therefore everything I share will always be free. I'm not interested in profiting off the knowledge of others.

Sharing relevant and helpful content is something I've always thought is a great approach to support the tech community. I'll keep offering my viewers insightful content. Thank you for supporting.

Did you find this article valuable?

Support Adit Modi by becoming a sponsor. Any amount is appreciated!

ย