Senior Cloud Data Engineer

As a Senior Cloud Data Engineer, you will play a key role in designing, developing, and optimizing cloud-agnostic data solutions, with a strong focus on Azure. Your expertise in Databricks, Spark, Lakehouse architectures, Delta Lake tables, and modern data engineering technologies will be essential in building and maintaining robust data pipelines.

This senior role requires technical excellence, leadership, and the ability to mentor junior engineers while independently tackling complex data engineering challenges.

Key Responsibilities

• Architect, build, and optimize scalable cloud-agnostic data solutions using Azure, Databricks, Spark, Lakehouse, and Delta Lake tables.
• Develop, implement, and maintain big data pipelines for ingesting, processing, and storing structured and unstructured data.
• Manage and optimise data lake and data warehouse architectures for performance, cost, and scalability.
• Work within Azure environments (Azure Synapse, Data Factory, ADLS) to develop and maintain cloud-based data solutions.

• Implement DevOps best practices, CI/CD pipelines, and automation.
• Ensure security, compliance, and governance in all data processes.
• Utilize big data tools for processing large datasets efficiently.
• Write advanced Python and T-SQL scripts for data transformation and ETL/ELT processes.
• Collaborate with stakeholders to develop data-driven solutions.
• Mentor junior engineers and lead technical initiatives.

Required qualifications and skills

• Degree in Computer Science or a related field, or equivalent data engineering certification.
• Eight-plus years of experience in cloud-based data engineering.
• Extensive experience with Azure (Azure Synapse, Data Factory, ADLS, etc.).
• Strong expertise in Databricks, Spark, Delta Lake, and Lakehouse architectures.
• Advanced proficiency in T-SQL and Python for complex data transformations and optimizations.
• Hands-on experience with big data processing, ETL/ELT, and data pipeline orchestration.
• Solid understanding of data modeling, warehouse design, and data Lakehouse architecture.
• Experience with Docker and Kubernetes for data processing workloads. (Advantageous)
• Knowledge of data security, encryption, and compliance regulations. (Advantageous)
• Exposure to machine learning and AI pipelines within data engineering workflows.
• Experience with multi-cloud environments (AWS, GCP in addition to Azure). (Advantageous)

The Reference Number for this position is NG60322 which is a Remote role offering a salary of R800,000 up to R1m per annum salary negotiable based on experience. E-mail Nokuthula on az.oc.egreM-e@galuhtukon or call her for a chat on 011 463 3633 to discuss this and other opportunities.

Are you ready for a change of scenery? E-Merge IT recruitment is a niche recruitment agency. We offer our candidates options so that we can successfully place the right people with the right companies, in the right roles. Check out the E-Merge IT website www.e-merge.co.za for more great positions.

Do you have a friend who is a developer or technology specialist? We pay cash for successful referrals!

Posted on 08 Apr 09:37, Closing date 7 Jun