Package: aws-analytics-reference-architecture
AWS Analytics Reference ArchitectureThe AWS Analytics Reference Architecture is a set of analytics solutions put together as end-to-end examples.
It regroups AWS best practices for designing, implementing, and operating analytics platforms through different purpose-built patterns, handling common requirements, and solving customers' challenges.This project is composed of:Reusable core components exposed in an AWS CDK (Cloud Development Kit) library currently available inTypescriptandPython. This library containsAWS CDK constructsthat can be used to quickly provision analytics solutions in demos, prototypes, proof of concepts and end-to-end reference architectures.Reference architectures consumming the reusable components to demonstrate end-to-end examples in a business context. Currently, theAWS native reference architectureis available.This documentation explains how to get started with the core components of the AWS Analytics Reference Architecture.Getting startedAWS Analytics Reference ArchitectureGetting startedPrerequisitesInitialization (in Python)DevelopmentDeploymentCleanupAPI ReferenceContributingLicense SummaryPrerequisitesCreate an AWS accountThe core components can be deployed in any AWS regionInstall the following components with the specified version on the machine from which the deployment will be executed:Python [3.8-3.9.2] or TypescriptAWS CDK v2: Please refer to theGetting startedguide.Bootstrap AWS CDK in your region (hereeu-west-1). It will provision resources required to deploy AWS CDK applicationsexportACCOUNT_ID=$(awsstsget-caller-identity--queryAccount--outputtext)exportAWS_REGION=eu-west-1
cdkbootstrapaws://$ACCOUNT_ID/$AWS_REGIONInitialization (in Python)Initialize a new AWS CDK application in Python and use a virtual environment to install dependenciesmkdirmy_democdmy_demo
cdkinitapp--languagepython
python3-mvenv.envsource.env/bin/activateAdd the AWS Analytics Reference Architecture library in the dependencies of your project. Updaterequirements.txtaws-cdk-lib==2.51.0
constructs>=10.0.0,<11.0.0
aws_analytics_reference_architecture>=2.0.0Install The Packages viapippython-mpipinstall-rrequirements.txtDevelopmentImport the AWS Analytics Reference Architecture in your code inmy_demo/my_demo_stack.pyimportaws_analytics_reference_architectureasaraNow you can use all the constructs available from the core components library to quickly provision resources in your AWS CDK stack. For example:The DataLakeStorage to provision a full set of pre-configured Amazon S3 Bucket for a data lake# Create a new DataLakeStorage with Raw, Clean and Transform buckets configured with data lake best practicesstorage=ara.DataLakeStorage(self,"storage")The DataLakeCatalog to provision a full set of AWS Glue databases for registring tables in your data lake# Create a new DataLakeCatalog with Raw, Clean and Transform databasescatalog=ara.DataLakeCatalog(self,"catalog")The DataGenerator to generate live data in the data lake from a pre-configured retail dataset# Generate the Sales Datasales_data=ara.BatchReplayer(scope=self,id="sale-data",dataset=ara.PreparedDataset.RETAIL_1_GB_STORE_SALE,sink_object_key="sale",sink_bucket=storage.raw_bucket,)# Generate the Customer Datacustomer_data=ara.BatchReplayer(scope=self,id="customer-data",dataset=ara.PreparedDataset.RETAIL_1_GB_CUSTOMER,sink_object_key="customer",sink_bucket=storage.raw_bucket,)Additionally, the library provides some helpers to quickly run demos:# Configure defaults for Athena consoleathena_defaults=ara.AthenaDemoSetup(scope=self,id="demo_setup")# Configure a default role for AWS Glue jobsara.GlueDemoRole.get_or_create(self)DeploymentDeploy the AWS CDK applicationcdkdeployThe time to deploy the application is depending on the constructs you are usingCleanupDelete the AWS CDK applicationcdkdestroyAPI ReferenceMore contructs, helpers and datasets are available in the AWS Analytics Reference Architecture. See the full API specificationhereContributingPlease refer to thecontributing guidelinesandcontributing FAQfor details.License SummaryThe documentation is made available under the Creative Commons Attribution-ShareAlike 4.0 International License. See the LICENSE file.The sample code within this documentation is made available under the MIT-0 license. See the LICENSE-SAMPLECODE file.