M23 - Data Engineer

Overview

Join the Digital Excellence & Products Division (DXD) within the Ministry of Education (MOE), where technology, data, and education come together to build impactful digital solutions. As a Data Engineer in the Geospatial Team, you will design, develop, and maintain scalable data platforms and pipelines that power geospatial analytics and spatial modelling. Your work will support evidence-based planning by enabling insights into future education demand, school locations, infrastructure planning, and resource optimisation.

Responsibilities

Data Engineering & Solution Design

• Collaborate with business users, data analysts, and stakeholders to gather requirements and translate them into scalable data solutions.
• Design technical architectures that align with enterprise data strategy, governance, and security standards.
• Develop robust and maintainable data models to support analytics and reporting.

Data Pipeline Development

• Design, build, test, and deploy end-to-end data pipelines for batch and real-time data processing.
• Develop data ingestion, transformation, validation, and orchestration workflows using modern data engineering frameworks.
• Implement automated data quality checks, error handling, and monitoring processes.

Data Platform & Infrastructure

• Design and optimise data architectures across Data Lakes, Data Warehouses, Lakehouses, and related platforms.
• Monitor, maintain, and optimise data infrastructure to ensure scalability, reliability, and cost efficiency.
• Troubleshoot pipeline failures, resolve data quality issues, and continuously improve platform performance.

Collaboration & Continuous Improvement

• Work closely with cross-functional teams to deliver data-driven solutions that support business objectives.
• Ensure compliance with data governance, metadata management, lineage, and security standards.
• Evaluate and recommend new technologies, tools, and best practices to modernise the data platform.

Requirements

Experience

• Minimum 3–5 years of experience in Data Engineering, Data Analytics, or a related technical field.
• Proven experience designing, building, and maintaining production-grade data pipelines and analytics platforms.

Technical Skills

• Strong proficiency in Python, including libraries such as Pandas and NumPy.
• Advanced SQL skills for data querying, transformation, and database management.
• Solid understanding of modern data architecture concepts, including:

1. Data Lake
2. Data Warehouse
3. Data Lakehouse
4. Data Mesh

• Experience with data ingestion, transformation, orchestration, and data quality management.
• Knowledge of cloud platforms, preferably AWS, and experience with Databricks is advantageous.
• Familiarity with infrastructure-as-code, DevOps practices, and cloud-native data platforms.
• Understanding of data governance, metadata management, and data lineage principles.
• Experience with GIS technologies such as ArcGIS, geospatial APIs, routing engines, or 2D/3D mapping technologies is a plus.

Soft Skills

• Strong analytical and problem-solving abilities with the capability to troubleshoot complex data engineering challenges.
• Excellent communication skills with the ability to translate business requirements into technical solutions.
• Strong stakeholder management and collaboration skills across technical and non-technical teams.
• Self-motivated, proactive, and committed to continuous learning and improvement.
• Ability to work effectively in Agile, cross-functional environments while delivering scalable, high-quality data solutions.