Job Openings G03 - Data Engineer

About the job G03 - Data Engineer

What the products are

We are seeking a Data Engineer to join our Data and Artificial Intelligence Platforms (DAIP) team. As the data technology landscape is rapidly evolving and data/system requirements are becoming increasingly complex and interconnected, to better support agencies, the next phase of GDA (i.e. GDA 2.0) aims to consolidate, streamline, and elevate the products and initiatives within the WOG data landscape this includes the development of the following products.

What the role is

As a data engineer, you will:

  • Architect and build ingestion pipelines to collect, clean, merge, and harmonize data from different source systems.
  • Day-to-day monitoring of databases and ETL systems, e.g., database capacity planning and maintenance, monitoring, and performance tuning; diagnose issues and deploy measures to prevent recurrence; ensure maximum database uptime;
  • Construct, test, and update useful and reusable data models based on data needs of end users.
  • Design and build secure mechanisms for end users and systems to access data in data warehouse.
  • Research, propose and develop new technologies and processes to improve agency data infrastructure.
  • Collaborate with data stewards to establish and enforce data governance policies, best practices and procedures.
  • Maintain data catalogue to document data assets, metadata and lineage.
  • Implement data quality checks and validation processes to ensure data accuracy and consistency.
  • Implement and enforce data security best practices, including access control, encryption, and data masking, to safeguard sensitive data.
  • Learn and share knowledge in a multi-disciplinary team.

Additionally, more senior engineers will be expected to:

  • Establish best practices
  • Share your expertise and mentor other engineers
  • Help with recruiting.

You are not just here to write code, but also to figure out what we should be building and how we should build it.

Your job will be to bring expertise and capability to the public sector. Sometimes this means coding new systems from scratch. Other times this means using the best solutions the community has to offer. We use cloud services, open source software, and commodity hardware as far as possible. Knowing what to build and what to reuse lets us avoid wasting time on solved problems and focus on delivering actual value.

What it is like working here

We build products that serve a variety of agency users, who use them to solve highly meaningful problems pertinent to our society, from transportation, to education, to healthcare. The public sector is full of opportunities where even the simplest software can have a big impact on peoples lives. We are here to improve how we live as a society through what we can offer as a government.

  • Rapid Prototyping - Instead of spending too much time debating ideas we prefer testing them. This identifies potential problems quickly, and more importantly, conveys what is possible to others easily.
  • Reliable Productization - To scale an idea, a prototype or a Minimum Viable Product to a software product, we scrutinize and commit to its usability, reliability, scalability and maintainability.
  • Ownership - In addition to technical responsibilities, this means having ideas on how things should be done and taking responsibility for seeing them through. Building something that you believe in is the best way to build something good.
  • Continuous Learning - Working on new ideas often means not fully understanding what you are working on. Taking time to learn new architectures, frameworks, technologies, and even languages is not just encouraged but essential.

What we are looking for

We work mostly in Python, JavaScript and Scala. We are looking for proficiency in at least one language and the ability to learn. Strong passion in software engineering is what matters to us.

We look for people who:

  • Have a demonstrated ability to build software
  • Can write code to solve abstract problems
  • Can think critically on how to get the code correct and cover the edge cases
  • Can talk and reason about code with other engineers
  • Have a demonstrated ability in writing efficient code
  • Adequate exposure to cloud or on-prem production environment, and experience in deployment would be an advantage
  • Understanding of devops, CI/CD would be an advantage
  • Have an interest in data science and machine learning
  • Take the initiative to make things happen
  • Want to work for the public good

Or refer someone