Data Engineer

Our client is seeking a highly skilled Data Engineer to join their DataWorks team. The successful candidate will be responsible for designing, building, and maintaining scalable data pipelines and modern data platforms that enable advanced analytics and data-driven decision-making. This role requires strong experience working with Microsoft Fabric, AWS, and Databricks, along with the ability to integrate data from multiple sources into efficient and reliable data architectures.

You will collaborate closely with data analysts, data scientists, and business stakeholders to ensure high-quality, accessible data that supports reporting, analytics, and machine learning initiatives.

Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes to support analytics and reporting requirements.
• Build and manage cloud-based data solutions using platforms such as AWS, Microsoft Fabric, or Databricks.
• Integrate data from multiple internal and external sources into centralized data platforms, data lakes, or data warehouses.
• Develop and maintain data models, data marts, and data warehouse structures to support business intelligence and analytics initiatives.
• Ensure data quality, integrity, and reliability through validation, monitoring, and automated quality checks.
• Work with large datasets and big data technologies such as Spark to process and transform complex data efficiently.
• Collaborate closely with data analysts, BI developers, data scientists, and business stakeholders to understand data requirements and deliver scalable solutions.
• Optimize data processing workflows and improve performance and scalability of existing data pipelines.

Qualifications and experience:

• Bachelor’s degree from an accredited university. A degree in computer science, engineering, information technology, or a related field is advantageous.
• Minimum of 7+ years of hands-on experience in data engineering or data platform development. Experience within the financial services industry would be beneficial but is not essential.
• Proven experience building, managing, and optimizing data pipelines and data solutions using on-premise or cloud-based technologies.
• Practical experience working with AWS services such as Glue, Redshift, S3, Lambda, EMR, and Athena, or exposure to Microsoft fabric or Databricks.
• Familiarity with big data frameworks and technologies such as Apache Spark, Hadoop, or Kafka.
• Strong programming capabilities in Python, Pandas, and SQL for data processing and transformation

The reference number for this position is NG60984 which is a Permanent, Hybrid role in Johannesburg offering a salary of up to R1.2mil per annum negotiable based on experience. E-mail Nokuthula on e-merge.co.za or call her for a chat on 011 463 3633 to discuss this and other opportunities.

Are you ready for a change of scenery? E-Merge IT recruitment is a niche recruitment agency. We offer our candidates options so that we can successfully place the right people with the right companies, in the right roles. Check out the e-Merge IT website www.e-merge.co.za for more great positions.

Posted on 26 May 14:30, Closing date 25 Jul