Data Engineer

Responsibilities

• Deploy, manage, and support data pipelines in production environments.
• Build and maintain scalable data processing solutions on AWS cloud platforms.
• Automate testing, deployment, and monitoring of data pipelines through CI/CD practices.
• Ensure reliability, performance, and operational stability of data platforms and workflows.
• Collaborate closely with Data Engineers, Machine Learning Engineers, Software Engineers, Product Managers, and domain experts to deliver data-driven solutions.
• Troubleshoot production issues, perform root cause analysis, and implement preventive measures.
• Follow Agile development practices and contribute to continuous improvement initiatives.

Requirements

• Minimum 5 years of experience deploying and managing data pipelines in production environments.
• Proven experience working in cross-functional teams comprising Data Engineers, Machine Learning Engineers, Software Engineers, Product Managers, and business stakeholders.
• Strong proficiency in SQL and relational database technologies.
• Hands-on experience with at least one workflow orchestration tool such as Airflow or Dagster.
• Strong Python programming skills, including experience with data processing libraries such as Pandas.
• Experience with containerization technologies, particularly Docker.
• Familiarity with CI/CD tools and practices, preferably Jenkins.
• Experience working in Agile development environments.

Preferred Skills

• Hands-on experience with AWS services such as RDS, EKS, EMR, and Redshift.
• Experience with Snowflake data platforms.
• Experience developing and deploying microservices using Flask and/or FastAPI.
• Knowledge of big data technologies such as Spark, Hadoop, and Kafka.
• Familiarity with SQLAlchemy and Alembic.
• Experience supporting large-scale data platforms, analytics, or machine learning workloads.

Nice to Have

• Exposure to MLOps, DataOps, or cloud-native data platform architectures.
• Experience implementing infrastructure automation and observability solutions.
• Knowledge of security and governance best practices within cloud-based data environments.