Data Engineer – Azure Databricks & Python (Remote)

About the role

We are looking for a Data Engineer to design and develop scalable data pipelines using Azure Databricks and Python. You will integrate AI and large‑language‑model (LLM) capabilities into data workflows, enabling automated classification and insight generation.

Key responsibilities

• Design, build, and optimise end‑to‑end data pipelines and Lakehouse architectures on Azure Databricks.
• Integrate LLM APIs and machine‑learning models into data processing streams for automated classification.
• Develop and maintain high‑quality Python code and SQL procedures for data transformation and ingestion.
• Collaborate with AI and product teams to deliver data‑driven solutions that create business value.
• Monitor, troubleshoot, and ensure the reliability and scalability of pipelines in distributed cloud environments.
• Implement data‑governance and security best practices to maintain platform integrity and compliance.

Required profile

• 4+ years of experience in Data Engineering, Machine Learning Engineering, or Software Engineering.
• Proven expertise in developing and optimising pipelines with Azure Databricks and Python.
• Experience integrating ML/AI solutions and LLM APIs into automated workflows.
• Strong background in data manipulation, automation, and classification tasks.
• Deep understanding of cloud‑native data architectures and AI integration patterns.
• Advanced proficiency in English.

Required skills

• Azure Databricks
• Python
• SQL
• LLM APIs
• Machine‑learning model integration
• Cloud‑native data architecture design
• Data‑governance practices

What we offer

• 100% remote work from anywhere.
• Competitive compensation in USD or local currency.
• Hardware and software setup for home office.
• Flexible schedule – create your own hours.
• Paid parental leave, vacations, and national holidays.
• Innovative, multicultural environment with top‑tier global talent.
• Mentorship, promotion pathways, and continuous skill development.