AI Data Engineer (Agentic AI & Large-Scale Data)

We are seeking a highly capable Data Engineer with strong experience in scalable data platforms, large-scale scraping, LLM integrations, and agentic AI systems. This role is ideal for a hands-on builder who enjoys solving complex data challenges and developing intelligent automation solutions that operate reliably in production environments.

You will play a key role in designing and building modern data pipelines, AI-powered workflows, and autonomous systems that can interact with tools, APIs, databases, and enterprise data sources.

Responsibilities:

• Design and build scalable batch and near-real-time data pipelines.
• Ingest and process data from APIs, databases, files, event streams, and web sources.
• Develop and maintain large-scale scraping and data acquisition systems.
• Clean, validate, transform, and model data for analytics and AI applications.
• Build LLM-powered workflows and agentic AI solutions using:
• Tool calling
• Structured outputs
• Retrieval-Augmented Generation (RAG)
• API and database integrations
• Support modern data lake, warehouse, and lakehouse architectures.
• Implement monitoring, observability, schema validation, and data quality controls.
• Collaborate with engineering teams to deliver secure, scalable, and maintainable production systems.

Qualifications and experience

• Strong experience with Python and/or Node.js/TypeScript.
• Advanced SQL and relational database skills.
• Proven experience building production-grade data pipelines.
• Experience with AWS or similar cloud platforms.
• Hands-on experience with data lakes, warehouses, or lakehouse technologies.
• Practical experience using LLM platforms such as OpenAI, Anthropic, Gemini, Bedrock, or similar.
• Experience building agentic workflows involving:
• Tool use
• Task decomposition
• Retrieval systems
• Memory
• Human-in-the-loop workflows
• Experience with scraping frameworks such as Playwright, Puppeteer, Scrapy, or Selenium.
• Strong understanding of APIs, containers, CI/CD, testing, and secure engineering practices.

The Reference Number for this position is NG61354 which is a Contract Remote role offering a salary of up to R550 per hour salary negotiable based on experience. E-mail Nokuthula on [email protected] or call her for a chat on 011 463 3633 to discuss this and other opportunities.

Are you ready for a change of scenery? E-Merge IT recruitment is a niche recruitment agency. We offer our candidates options so that we can successfully place the right people with the right companies, in the right roles. Check out the E-Merge IT website www.e-merge.co.za for more great positions.

Do you have a friend who is a developer or technology specialist? We pay cash for successful referrals

Posted on 25 May 09:52, Closing date 24 Jul