Job Openings
Azure Data Engineer
About the job Azure Data Engineer
This role will be responsible for ensuring the scalability, reliability, and security of data processing frameworks while supporting legacy systems and modern cloud-based architecture
Responsibilities:
- Design, develop, and maintain scalable data pipelines using Microsoft Fabric (OneLake, Delta Lake, Synapse, Data Pipelines).
- Migrate legacy pipelines from Azure Data Factory to Microsoft Fabric Data Pipelines.
- Build efficient ETL/ELT workflows to process and transform data from structured and semi-structured sources.
- Optimize data storage, partitioning, and retrieval strategies for improved performance and cost efficiency.
- Implement CI/CD pipelines for data workflows, ensuring seamless deployment and integration of data models and transformations.
- Monitor and troubleshoot data pipeline performance, ensuring reliability and data integrity.
- Collaborate with BI analysts and business stakeholders to ensure efficient data delivery for reporting and analytics.
- Support and maintain legacy databases (PostgreSQL, MySQL, etc.), ensuring smooth integration with Microsoft Fabric.
- Implement data security, compliance, and governance best practices.
Requirements:
- 4+ years of experience as a Data Engineer, with expertise in cloud-based data solutions.
- Strong knowledge of Microsoft Fabric, Azure Synapse, OneLake and Delta Lake.
- Proficiency in SQL, Python, Spark, and data transformation frameworks.
- Experience building ETL/ELT workflows using Azure Data Factory, Fabric Data Pipelines.
- Microsoft Purview, DBT and Terraform knowledge is nice to have.
- DevOps expertise, including CI/CD pipelines, Infrastructure as Code (IaC), and Git-based version control.
- Strong understanding of data warehousing, performance tuning, and data partitioning strategies.