Job Openings Senior Data Engineer Consultant

About the job Senior Data Engineer Consultant

Mission Overview:

Keystone Solutions is seeking a Senior Data Engineer for a consultancy mission at a client site. You will be part of a dynamic team working on the Data Exchange Platform (DEP) project, which involves building an on-premise data platform for a governmental organization. This role emphasizes the importance of security, governance, and data sovereignty.


Main Responsibilities:

  • Design and deploy data ingestion pipelines from heterogeneous sources: files, APIs, relational databases.
  • Implement pipelines for incremental and streaming ingestion.
  • Ensure reliability, idempotence, and traceability of each pipeline.
  • Develop transformation models on Bronze, Silver, and Gold layers (Medallion architecture).
  • Implement best practices for data historization (e.g., SCD2, upsert, append only).
  • Develop exports to Delta Lake.
  • Manage and maintain Delta Lake tables.
  • Guarantee transactional consistency of data in a multi-domain environment.
  • Implement and maintain orchestration assets.
  • Monitor runs, manage errors, and automate alerts.
  • Push technical and business metadata to the data platform.
  • Maintain end-to-end lineage.
  • Contribute to the drafting of DUA, SLA, and associated governance documents.
  • Develop endpoints to expose the Gold layer to BI dashboards.
  • Collaborate with reporting teams for BI dashboard feeding.

Required Technical Skills:

  • Proficiency in Python.
  • Proficiency in SQL.
  • Proven experience with dbt-core (models, snapshots, macros, tests, profiles).
  • Good knowledge of DuckDB as an embedded analytical query engine.
  • Experience with Delta Lake (ACID transactions, time travel, OPTIMIZE/VACUUM).
  • Knowledge of relational databases: MSSQL, PostgreSQL (advanced SQL, CDC).
  • Experience with a data orchestrator: Dagster, Airflow, or equivalent.
  • Comfortable with Linux/OpenShift and using PowerShell (Windows dev environment).
  • Familiarity with governance tools: DataHub, OpenMetadata, or equivalent.
  • Experience in BI development.

Preferred Skills:

  • Experience with Kafka/Debezium for real-time data capture.
  • Knowledge of Spark (Spark SQL, Thrift Server, Beeline).
  • Experience with Power BI Report Server (PBIRS) or Apache Superset.
  • Awareness of data security in restricted access environments.
  • Knowledge of Lakehouse concepts (Medallion architecture), DWH, and Datalake.

Soft Skills:

  • Rigorous and autonomous in managing complex multi-domain pipelines.
  • Ability to document technical decisions and simplify for non-technical stakeholders.
  • Team spirit.
  • Curious and solution-oriented mindset.

What We Offer:

  • A stable mission that is long-term.
  • A high-impact project in a secure and technically stimulating environment.
  • A modern and open-source stack: DLT, dbt, DuckDB, Delta Lake, Dagster.
  • An experienced and supportive team focused on quality and best practices.
  • Opportunities for skill enhancement in governance, orchestration, and lakehouse.

If you are ready to tackle technical and strategic challenges in a dynamic consultancy environment, apply today at Keystone Solutions Career Portal.


Duration: As soon as possible - 31/12/2026 5 months • (full time)


Skills required:

  • Dagster - Level: Junior - Most recent: Any time
  • Data acquisition (ETL, ELT, ...) - Level: Confirmed - Most recent: Any time
  • Data architecture - Level: Junior - Most recent: Any time
  • data engineering - Level: Confirmed - Most recent: Any time
  • Data Governance - Level: Confirmed - Most recent: Any time
  • DataHub - Level: Junior - Most recent: Any time
  • datalake - Level: Junior - Most recent: Any time
  • dbt - Level: Junior - Most recent: Any time
  • Kafka - Level: Junior - Most recent: Any time
  • Metadata management - Level: Confirmed - Most recent: Any time
  • POWER BI - Level: Junior - Most recent: Any time
  • Python (from a Data Engineer perspective), Pandas & Apache Spark - Level: Confirmed - Most recent: Any time
  • SPARK - Level: Junior - Most recent: Any time
  • SQL - Level: Confirmed - Most recent: Any time

Language requirements:

Dutch
Level Active knowledge
English
Level Active knowledge
French
Level Native


Or refer someone