Intermediate Data Engineer

Minimum requirements:

• Post Graduate Degree: Information Technology
• Post Graduate Degree: Information Studies
• Masters Degree: Information Technology
• Masters Degree: Information Studies

Experience

• Minimum 3 years experience
• IT Architecture
• Data Integrity
• IT Applications
• Data Analysis
• Knowledge Classification
• C++.
• Amazon Web Services.
• Amazon S3.
• Databricks

Skills:

• Database systems (SQL and NoSQL)
• Data engineer must know how to manipulate database management systems (DBMS), which is a software application that provides an interface to databases for information storage and retrieval
• Data warehousing solutions
• ETL tools
• ETL (Extract, Transfer, Load)
• Data APIs
• Allows two applications or machines to communicate with each other for a specified task
• Python, Java, R and Scala programming languages
• Understanding the basics of distributed systems

Responsibilities:

• Identify shortcomings and suggest improvements to existing processes, systems and procedures, then delivers a plan for a small element of a change management program with guidance from a project/program manager
• Build analytics tools that utilise the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics
• Monitor the existing metrics, analyse data, and lead partnership with other Data and Analytics teams in an effort to identify and implement system and process improvements
• Utilise data to discover tasks that can be automated and identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc
• Developing ETL processes that convert data into formats for consumption
• Risk, Regulatory, Prudential and Compliance
• Responsible for executing testing and validation in line with data governance and quality business requirements
• Liaise with and collaborate with data analysts, data warehousing engineers, and data scientists in finding and applying best practices within the Data and Analytics department as well as defining the business data requirements, which will ensure that the collected data is of high quality and optimal for use across the department and the business at large
• Acts as a subject matter expert from a data perspective and provides input into all decisions relating to data engineering and the use thereof. 
• Provide guidance in terms of setting, governance standards