Data Warehouse Automation Using dbt
For the development of this data warehouse, we are utilizing AWS Glue services to read data from SAP and transfer it to an S3 bucket during the first stage. In the second stage, we are retrieving the data from the S3 bucket and transferring it to Snowflake using dbt. All data transformation processes are being performed using dbt, and the final data is being stored in the Snowflake data warehouse for Power BI reporting purposes.
Technology Stack
- Technology: dbt, SAP,AWS Glue, AWS S3, Power BI
- Database: Snowflake
- Project date: 17 August, 2021