Home

Welcome! 👋

Data Engineer

I’m Pradeep Kalluri, a Data Engineer specializing in building scalable cloud data platforms and production-grade data pipelines.

Currently at NatWest Bank in London, designing and delivering reliable data engineering solutions that power analytics and business intelligence across the organization.

💡 What I Do

I build end-to-end data platforms that transform raw data into actionable insights:

  • Data Ingestion - Real-time streaming (Kafka) and batch processing from cloud storage (S3, Azure Data Lake)
  • Distributed Processing - Large-scale data transformation using PySpark and Databricks
  • Data Warehousing - Building curated datasets in Snowflake with optimized data models
  • Pipeline Orchestration - Workflow automation and monitoring with Apache Airflow
  • Analytics Engineering - dbt transformations, data quality frameworks, BI integration

🎯 Technical Stack

Languages: Python (PySpark, Pandas), SQL, Shell Scripting
Cloud Platforms: AWS (S3, Glue, Lambda), Azure (Databricks, ADF, Data Lake), Microsoft Fabric
Data Engineering: Apache Kafka, Apache Airflow, Snowflake, dbt, ETL/ELT pipelines
Databases: Snowflake, Azure SQL, Redshift, PostgreSQL, MySQL
DevOps: Docker, Terraform, CI/CD (GitHub Actions, Azure DevOps), Git
BI Tools: Tableau, Power BI

📊 Recent Achievements

📝 Technical Writing

Published on The New Stack ⭐ + 71,000+ views across platforms

Published on The New Stack, Apache Airflow Official Medium, cross-posted on Dev.to, and discussed on Reddit’s r/dataengineering

View all articles →

🎤 Speaking

Oxford Microsoft Data Platform Group - January 21, 2026 ✅ COMPLETED
Topic: “From Raw to Refined: Building Production Data Pipelines That Scale” Audience: 50+ registrations from industry leaders

Presented to data engineers with 50+ registrations for the event from industry leaders on production data pipeline architecture. Received positive feedback from Microsoft Senior Cloud Solution Architect and invited back for dedicated Apache Airflow session.

13 conference proposals submitted to data engineering conferences and meetups across Europe

View all speaking engagements →

💻 Open Source Contributions

Apache Airflow - 3 merged PRs + 5+ PR reviews completed:

  • Data masking documentation (PR #58587) - ✅ MERGED
  • Pool name validation fix (PR #59938) - ✅ MERGED
  • Bug fix contribution (PR #61005) - ✅ MERGED

dbt-core - 1 merged + 2 active contributions:

  • Fixed @requires.catalogs decorator for compile command (PR #12388) - ✅ MERGED (Feb 2026)
  • dbt init UX fix (PR #12232) - 🟡 Under review
  • Debug compilation error fix (PR #12502) - 🟡 Under review

🏅 Certifications

Microsoft Fabric Data Engineer Associate — January 4, 2026 View credential

SnowPro Core (COF-C03) — Score: 923/1000 — February 16, 2026 View credential

🚀 Currently

  • Building production data pipelines at NatWest Bank processing millions of transactions daily
  • Writing about data engineering on Medium (71K+ views) and Dev.to
  • Contributing to Apache Airflow (3 merged PRs) and dbt-core open source projects
  • Speaking at data engineering meetups and conferences (Oxford Microsoft Data Platform Group - Jan 2026)
  • Mentoring data engineers on Topmate — “The Career Launcher” service
  • Pursuing UK Global Talent Visa in Digital Technology

🏢 Professional Experience

NatWest Bank (Sep 2025 - Present) - Data Engineer
Building scalable data platforms with Kafka, PySpark, Snowflake, and Airflow

Accenture (Jul 2023 - Aug 2025) - Data Engineer
Delivered enterprise cloud data solutions across Azure and AWS for major clients

Dpoint Group (May 2022 - Jun 2023) - Data Engineer
Developed BI solutions and ETL pipelines supporting operational analytics

View detailed experience →

🔗 Connect With Me

Email • LinkedIn • GitHub • Medium • Dev.to

📍 Based in London, United Kingdom

Passionate about building reliable, scalable data platforms that empower data-driven decision making.