Job description:

• Design, build and maintain scalable data pipelines and backend services
• Support real-time decisioning, analytics, reporting and data collection
• Translate business/data requirements into appropriate data models and technical solutions
• Manage data transformation, integration and processing
• Develop high-quality, secure and well-tested code
• Build solutions that improve data governance, security and quality
• Implement processes and controls around Data Security and Data Quality
• Collaborate with developers and stakeholders while providing technical guidance

Requirements:

• 5+ years’ Data Engineering experience, particularly ETL/ELT, Data Warehousing and BI
• 5+ years building end-to-end data systems/pipelines using Python and/or Scala
• Strong SQL skills, including data analysis, troubleshooting and root-cause analysis
• Hands-on experience with Databricks, PySpark, Spark Streaming and Delta Lake
• Experience with large-scale datasets and complex data transformations
• Strong experience with cloud-based data platforms, ideally Microsoft Azure
• Experience with Azure Data Lake / Blob Storage or comparable cloud storage
• Good understanding of data modelling and data architecture principles
• Experience working within CI/CD environments
• Exposure to event-driven architectures such as Kafka, Event Hubs or Apache Flink
• Experience with Docker / Kubernetes / Helm and containerised environments
• Knowledge of dbt and BI/reporting tools
• Strong communication skills with the ability to provide technical guidance/leadership