Rahul

Rajasekharan

Building the modern data stack — dbt, Airflow, PySpark, AWS. Automating pipelines, improving DX, and turning repetitive work into clean, automated systems.

View Projects

Experience

2012 – Present · 14 yrs 2 mos

📍 London, UK

Direct Line Group (DLG)

▶ NOW

Sr. Data Engineer.

Feb'23–Now
AWS EcosystemdbtAirflowPySpark

Sainsbury's

Sr. Data Engineer.

2022–23
AWS EcosystemSnowflakedbtAirflow

Accenture

Sr. Data Engineer.

2021–22
AWS EcosystemPySpark

Cognizant

Data Engineer

2019–21
InformaticaAWS EcosystemPython
📍 Chennai, IN

Cognizant

Data Engineer

2018–19
InformaticaAWS EcosystemPython

TCS

Data Engineer

2012–18
InformaticaDatastageUnixHadoopPython

Skills

Production tools I use to design, orchestrate, and operate data platforms.

Languages & Core

PythonSQLBashPL/SQLExcel VBA

Databases

PostgreSQLRedshiftSnowflakeOracle

Processing & Orchestration

Apache AirflowSparkPandasdbtStep Functions

Gen AI & LLMs

OllamaAgnoLangchainVectorDBsRAG

Storage

S3HDFS

Infra

CloudFormationDockerTerraformGithub ActionsCode Pipeline

Quality & Governance

YData Profiling

Modeling Patterns

Star Schema3NFMedallion/Lakehouse

Ops & Practices

SLIs/SLOsCost OptimizationCI/CDCode Reviews

Certifications

Industry-recognised credentials across cloud and analytics.

AWS

AWS Certified Solutions Architect

AWS Practitioner

Airflow

Astronomer Certified DAG Authoring

Astronomer Certified Airflow Fundamentals

DBT

DBT Fundamentals

Snowflake

Snowflake Fundamentals

01

Self-Healing Data Pipelines with AI

PythonApache AirflowPostgreSQLpgvectorOllamaStreamlitDocker
02

DAG Schedule Visualisation & Scheduler Load Analysis

PythonApache AirflowJavaScriptObservabilityHeatmap
03

Data-Level Regression Detection for dbt

Pythondbt CorePostgreSQLRedshiftTyper CLIGit WorktreesDocker
04

Python-Driven UI Extensions for Airflow

PythonApache AirflowFastAPIReactMarkdownMermaid
05

The Airflow DAG Quality Auditor

PythonApache AirflowFastAPIReact
06

Cricket Analytics Platform — Snowflake + dbt + Airflow + Streamlit + Cortex

SnowflakedbtApache AirflowCosmosStreamlitCortex
07

Spark Job & Stage Execution Analyzer

PySparkScalaSparkListenerAWS Glue
08

LLM-Powered Automatic Documentation for dbt

dbtPythonLLMsOllamaOpenAI
09

Bulk Pause / Unpause DAGs from the UI

AirflowReactREST API
10

Intelligent Task Retries powered by LLMs

AirflowLLMsOllama
11

Documentation Assistant for Data Engineers

AirflowLLMs
12

AI Powered Personal Stylist

AirflowWeather APILLMs

Let's work together