Senior Data Engineering

Job Title: Senior Data Engineer

Job Overview

We are seeking an experienced Senior Data Engineer to design, develop, and optimize scalable data platforms and pipelines using Microsoft Fabric. This role will be responsible for building reliable data infrastructure, implementing modern Lakehouse architecture, and delivering high-quality data assets that support analytics and business intelligence.

The ideal candidate has strong hands-on expertise in SQL, Python/PySpark, cloud data platforms, data modeling, and Microsoft Fabric, with the ability to work closely with Data Analysts and BI Developers to deliver production-ready solutions.

Key Responsibilities

• Data Pipeline Development: Design, develop, and manage scalable end-to-end data pipelines and ETL/ELT workflows using Microsoft Fabric Data Factory, including Pipelines and Dataflows Gen2. Integrate data from various batch and streaming sources into OneLake.
• Lakehouse Architecture: Build and maintain Medallion Architecture across Bronze, Silver, and Gold layers within Microsoft Fabric. Utilize Lakehouses, Warehouses, CSV, and Delta/Parquet formats to establish a reliable single source of truth.
• Data Transformation & Modeling: Develop efficient data transformation processes using PySpark, Fabric Notebooks, dbt, and T-SQL. Design dimensional data models, including Star and Snowflake Schemas, optimized for analytical workloads and Power BI Direct Lake.
• Data Documentation & Lineage: Create and maintain technical documentation covering data flows, data lineage, Data Flow Diagrams (DFD), and Entity Relationship Diagrams (ERD) to ensure clear understanding of the data environment.
• Performance Optimization: Monitor and optimize Microsoft Fabric Capacity Unit (CU) consumption, Spark workloads, data processing, and query performance to ensure efficient platform utilization.
• Data Quality & Integration: Implement automated data quality controls and validation processes while ensuring seamless integration across the Microsoft Fabric ecosystem.
• Cross-Functional Collaboration: Work closely with Data Analysts, BI Developers, and other technology teams to develop clean, reliable, and production-ready data products that meet business and analytical requirements.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.
• 3–5+ years of professional experience in Data Engineering, BI Development, Data Platform Engineering, or a similar role.
• Strong hands-on experience in data pipeline development and cloud-based data platforms.
• Advanced proficiency in SQL and Python/PySpark.
• Strong knowledge of:
  • Data Lakehouse and Medallion Architecture
  • Data Warehousing
  • Dimensional Modeling
  • Fact and Dimension table design
• Practical experience with Microsoft Fabric, including:
  • OneLake
  • Lakehouse
  • Warehouse
  • Data Factory
  • Fabric Notebooks
• Candidates with experience in equivalent Azure Data technologies such as Azure Synapse, Azure Data Factory, Azure Databricks, or ADLS Gen2 will also be considered.

Preferred Qualifications

• Microsoft certifications such as:
  • DP-600 – Microsoft Certified: Fabric Analytics Engineer Associate
  • DP-700 – Microsoft Certified: Fabric Data Engineer Associate
• Knowledge of Data Governance and Data Security, including Row-Level Security (RLS) and Object-Level Security (OLS).
• Familiarity with Microsoft Purview and its integration with modern data platforms.
• Strong analytical, problem-solving, and troubleshooting skills with the ability to optimize data solutions for scalability and performance.