Docker Compose environments for developing modern data platform architectures using Kafka, Flink, Spark, Iceberg, OpenLineage, OpenMetadata, Pinot, ClickHouse, StarRocks + Kpow & Flex by Factor House

Tools for processing sql queries metadata.

Covid19 and Iowa Liquor Sales analysis at BigQuery using dbt, Airflow, Marquez, Google Cloud and other modern data stack tools

NextGen DataMotion Lineage

Simple python script for easy local airflow deployment with docker. Packed with additional components. Will be adding more going forward.

This Project aims to enhance Spark Lineage Generation in Open Metadata by creating Lineage edges from Table-to-Table with Column level lineage as well as Container to Table Lineage.

Testing a Docker deployment of Marquez and OpenLineage

Incident correlation engine for data teams — connects test failures to job runs via OpenLineage

Exploring how to surface dlt pipeline metadata via Airflow's OpenLineage integration.

dbt plugin that accelerates incident resolution with automated test failure correlation