Data Engineer-L6Y4V33W

Data Engineer

B2B or CIM

Remote

Discover us:

We are a global technology consulting firm dedicated to optimizing client productivity and profitability. We empower businesses, from mid-sized enterprises to international corporations, with innovative tech solutions.

We foster a culture of impact, enabling clients to excel through technology and innovation in their business and personal lives. Our commitment to team growth, risk-taking, and unwavering quality sets us apart, making us a company of excellence delivering exceptional results.

Job Overview:

We are seeking a highly skilled and driven Data Engineer to design, build, and maintain scalable data pipelines and infrastructures that fuel data-driven decision-making processes. You will collaborate with cross-functional teams, working on cutting-edge cloud platforms and distributed data processing frameworks.

If you are passionate about big data, cloud technologies, and delivering robust data engineering solutions at scale, this role is for you.

We are looking for professionals with prior experience in:

• Designing, developing, and maintaining scalable data pipelines using Python and SQL to handle large-scale data in real-time and batch processing.

• Implementing distributed data processing systems with Apache Spark and workflow orchestration using Apache Airflow.

• Implementing and managing ETL processes using Matillion.

• Managing and maintaining cloud-based data platforms using AWS and Azure Cloud technologies, integrating with Snowflake, Databricks, or other cloud-native services.

• Designing and implementing data quality frameworks using DBT, Great Expectations

• Collaborating with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.

• Excellent understanding of data warehousing concepts and best practices.

• Designing and implementing scalable data architecture, with a focus on modern frameworks such as Data Mesh for decentralized data ownership and Data Vault for flexible, scalable data modeling and storage solutions.

Key skills we are looking for:

• Programming: in Python for data acquisition and data processing.

• Database Expertise: Strong SQL skills for querying and manipulating data in RDBMS and NoSQL systems.

• Distributed Data Processing: Exposure to Apache Spark and DBT for high-volume data processing.

• Workflow Orchestration: Exposure to MS Fabric pipelines Azure Synapse/ADF, Matillion, Apache Airflow.

• Data Formats: Exposure to data formats such as Parquet, JSON, and Apache Iceberg.

• Cloud Platforms: expertise in AWS- Glue, Redshift, S3, DynamoDB and Azure -Databricks, Synapse, CosmosDB.

Nice-to-Have Skills:

• Streaming Technologies: Familiarity with Apache Flink and Kafka for real-time data processing.

• Data Governance: Experience with tools like Purview, Open Metadata or other similar solutions.

Why Join Us?

We offer exciting opportunities to work on multiple innovative projects that leverage the full spectrum of technologies mentioned in the job description, including Python, SQL, Apache Spark, Airflow, DBT, and cloud platforms like AWS and Azure. These projects span diverse industries, providing you with exposure to data engineering challenges and allowing you to apply your skills in real-world, impactful solutions.

We are committed to your long-term growth and success. Whether you're looking for a stable, long-term career path within our organization or prefer the flexibility of a contracting arrangement, we are open to finding the right fit for you. While our preferred setup is full-time employment, we are flexible and willing to explore alternative working formats to suit your needs.

Join us and become part of a collaborative and dynamic team that values innovation, continuous learning, and professional development. If you are passionate about building data-driven solutions and want to make a meaningful impact, we encourage you to apply!

Take the next step in your career and apply today! We look forward to discussing how your skills and experience can contribute to our exciting projects.