Azure Data Engineer

This role will be responsible for ensuring the scalability, reliability, and security of data processing frameworks while supporting legacy systems and modern cloud-based architecture

Responsibilities:

• Design, develop, and maintain scalable data pipelines using Microsoft Fabric (OneLake, Delta Lake, Synapse, Data Pipelines).
• Migrate legacy pipelines from Azure Data Factory to Microsoft Fabric Data Pipelines.
• Build efficient ETL/ELT workflows to process and transform data from structured and semi-structured sources.
• Optimize data storage, partitioning, and retrieval strategies for improved performance and cost efficiency.
• Implement CI/CD pipelines for data workflows, ensuring seamless deployment and integration of data models and transformations.
• Monitor and troubleshoot data pipeline performance, ensuring reliability and data integrity.
• Collaborate with BI analysts and business stakeholders to ensure efficient data delivery for reporting and analytics.
• Support and maintain legacy databases (PostgreSQL, MySQL, etc.), ensuring smooth integration with Microsoft Fabric.
• Implement data security, compliance, and governance best practices.

Requirements:

• 4+ years of experience as a Data Engineer, with expertise in cloud-based data solutions.
• Strong knowledge of Microsoft Fabric, Azure Synapse, OneLake and Delta Lake.
• Proficiency in SQL, Python, Spark, and data transformation frameworks.
• Experience building ETL/ELT workflows using Azure Data Factory, Fabric Data Pipelines.
• Microsoft Purview, DBT and Terraform knowledge is nice to have.
• DevOps expertise, including CI/CD pipelines, Infrastructure as Code (IaC), and Git-based version control.
• Strong understanding of data warehousing, performance tuning, and data partitioning strategies.