# Data Engineering Projects

## Real-Time Data Pipeline - NatWest Bank

**2025 - Present**

Building production data pipelines that ingest, process, and deliver analytics-ready data across the organization.

**Key Components:**

- Data ingestion from Amazon S3 using Kafka

- PySpark processing pipelines to raw and curated zones

- Snowflake data warehouse for downstream consumption

- Airflow DAG orchestration for workflow management

- Tableau dashboards for business intelligence

**Technologies:** Kafka, PySpark, Amazon S3, Snowflake, Airflow, Tableau, Python, SQL

**Impact:** Supports scalable analytics and reporting across multiple business units

## Enterprise Cloud Data Platform - Accenture

**2023 - 2025**

Delivered large-scale cloud data engineering solutions for multiple enterprise clients across Azure and AWS.

**Key Achievements:**

- Built end-to-end data platforms using Azure Databricks and Snowflake

- Developed ETL/ELT pipelines with PySpark and SQL

- Designed cloud-native data architectures

- Implemented data ingestion, transformation, and orchestration workflows

- Supported hybrid cloud solutions (Azure + AWS)

**Technologies:** Azure Databricks, Snowflake, ADF, Azure Data Lake, PySpark, Microsoft Fabric, AWS

**Impact:** Enabled data-driven decision making for enterprise clients

## Business Intelligence Data Flows - Dpoint Group

**2022 - 2023**

Developed BI and analytics solutions supporting operational reporting and KPI dashboards.

**Key Work:**

- Built ETL processes using SSIS

- Integrated SAP BW data into downstream systems

- Created automated data flows for analytics

- Supported business intelligence dashboards

**Technologies:** SSIS, SAP BW, SQL, Power BI

**Impact:** Improved reporting efficiency and data accessibility for business stakeholders

## Areas of Focus

- **Data Pipeline Architecture** - Designing scalable, reliable data flows

- **Cloud Data Engineering** - Azure and AWS platform expertise

- **Stream Processing** - Kafka, real-time data ingestion

- **Data Orchestration** - Airflow workflow management

- **Analytics Integration** - Connecting data platforms to BI tools

## Open Source \& Community

Actively contributing to the data engineering community through:

- Open source contributions

- Technical writing and documentation

- Knowledge sharing on best practices

[View my GitHub →](https://github.com/kalluripradeep)