Robust ETL Process with Airflow, Docker and Composer

Abstract

ETL (Extract, Transform and Load) is a method to process data to have a clean error free dataset. As part of creating a clean dataset, I have developed Python and R scripts to extract and transform the data and with the help of docker, composer and airflow scheduled jobs the data is loaded onto BigQuery. The data is then used to report or explore in Looker by creating LookML scripts and dashboards.

First Name
Milind
Last Name
Siddhanti
Industry
Organization
Supervisor
Date
Spring 2019