Data Engineer

Hi! We are DATA Group and we are searching for the best talent! Our goal is to simplify our clients' lives with innovative IT solutions. We operate at global scale and we are expanding to Portugal!

If you are passionate and have the desire to make the difference, we want to get to know you! Join us to be part of this incredible adventure!

Who are we looking for? YOU!

Location will be Lisbon, and the working model will be on-site/hybrid. Candidates must be living in Portugal.

Requirements:

Role and Responsibilities

• Uploading data to the Big Data Platform
• Targeted preparation of data for specific use cases and creation of data profiles
• Development of ready-to-use data pipelines
• Coordination of the service provider's activities (only complex)

Skills

• Proficient in the programming languages Apache Spark, Python, R, Scala, Java and Kotlin
• Understanding of AWS Cloud Ingestion solutions
• Understanding of efficient data processing and attitude in S3 and databases (on-prem and cloud)
• Knowledge of relevant source systems and their data types and structures
• Knowledge in the implementation of batch and real-time interfaces
• Extended expertise in the field of software engineering for the series production of productive data products (incl. testing, deployment, code quality, logging, monitoring, operation)
• Experience in the development of distributed systems
• Experience in Docker-based deployment and Resource Manager
• Experience in CI/CD pipelines and tools
• Knowledge and practical experience with Workflow Managers
• Detailed knowledge of agile software development according to Scrum
• Technical knowledge of the automotive industry

Years of Experience: 3 years