➡️ Build a data lake using AWS Lake Formation, then share it as a self-documented GraphQL API, automatically built with a reactive CI/CD pipeline, allowing authorized external clients to query your data lake using GraphQL syntax over HTTPS.
1) Set up a data lake using AWS Lake Formation and Amazon S3.
2) Crawl the data in the S3 buckets with AWS Glue to build a data catalog.
3) Monitor any changes in the AWS Glue Data Catalog with Amazon CloudWatch, and trigger an AWS Lambda function on every catalog change to update the schema of the external API.
4) In the Lambda function, query the latest data catalog and security settings, automatically generate source code to build and deploy a new version of the external API’s schema and resolvers, and commit the code to AWS CodeCommit.
5) Set up CodeCommit to automatically start a new CI/CD pipeline on AWS CodePipeline after any code commit.
6) Use CodePipeline to build a new version of the API with AWS CodeBuild using the newly-generated source code, and use AWS CodeDeploy to deploy both the new AWS AppSync GraphQL API and the new Lambda query resolvers.
7) Allow your GraphQL clients to securely invoke the external API over HTTPS with your chosen authentication mechanism. Active the AppSync server-side caching.
8) Use the AppSync Lambda query resolvers to translate the GraphQL query into SQL queries supported by Amazon Athena. 9) Run Athena jobs synchronously with the Athena-Express library, appropriately timing out and limiting the result set.
You can download the Implementation Guide from this link 👀👇 Link
I hope you will find it useful for your organization! 🤓
[IMPORTANT]
- This material is not mine. This material was written exclusively by Amazon Web Services.
I am sharing this with a bigger IT audience to help folks in their quest to learn AWS. I don't intend to monetize anything, therefore everything I share will always be free. I'm not interested in profiting off the knowledge of others.
Sharing relevant and helpful content is something I've always thought is a great approach to support the tech community. I'll keep offering my viewers insightful content. Thank you for supporting.