Warrendale, Pennsylvania, United States
Data Engineer
Job Description:
Our client is looking for a mid-level Data Engineer to assist in the design, build and maintenance of their cloud environment.
Key Responsibilities:
Primary Responsibilities Data Engineering (80%)
- Design, build, and maintain robust, scalable data pipelines using Databricks on cloud platforms (e.g., Azure or AWS).
- Develop and manage data models that support analytics, reporting, and operational use cases.
- Implement best practices for data lake architecture, ensuring high availability, performance, and data quality.
- Collaborate with analysts, data scientists, and business stakeholders to understand data requirements and deliver high-impact solutions.
- Automate data ingestion, transformation, and loading processes (ETL/ELT) using modern orchestration tools.
- Optimize and monitor pipeline performance and troubleshoot data-related issues proactively.
- Ensure compliance with data governance and security standards.
Secondary Responsibilities Cloud Data Environment Support (20%)
- Support the ongoing operations and enhancements of the organizations Cloud Data Environment.
- Assist with user onboarding, environment monitoring, and issue resolution.
- Collaborate with IT and cloud teams to ensure optimal integration and performance.
- Contribute to documentation, training, and knowledge-sharing related to the cloud data environment.
Qualifications:
- Bachelors or Masters degree in Computer Science, Engineering, Data Science, or a related field.
- 3+ years of experience in a data engineering or similar role.
- Proficiency with Databricks, Spark, and cloud data platforms (e.g., Azure Data Lake, AWS S3).
- Strong SQL and Python skills.
- Experience with data modeling, ETL/ELT pipeline design, and workflow orchestration tools (e.g., Airflow, dbt).
- Familiarity with data governance, security, and performance optimization.
- Experience supporting enterprise cloud data environments is a plus.