Data Engineer - Contract - Melbourne/Sydney

Requirements

• 5+ years of Data Engineering experience in enterprise environments.
  
• 3+ years of hands-on experience working with Azure Databricks and Apache Spark.
  
• Proven experience delivering data solutions on Azure cloud platforms.
  
• Experience designing and implementing scalable data pipelines and data lake architectures.
  
• Strong experience with Agile delivery methodologies.
  
• Ability to work independently and contribute immediately within a project-based environment.
  
• Strong stakeholder management and communication skills.
  

Key Responsibilities :

• Design, develop, and optimize data pipelines using Azure Databricks and Apache Spark.
  
• Build and maintain robust ETL/ELT frameworks to ingest, transform, and load data from multiple source systems.
  
• Develop scalable data solutions leveraging Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.
  
• Implement data quality, validation, and monitoring frameworks to ensure data integrity and reliability.
  
• Collaborate with solution architects, business analysts, data scientists, and stakeholders to deliver project outcomes.
  
• Optimize Spark workloads, cluster configurations, and data processing performance.
  
• Support deployment activities through CI/CD pipelines and DevOps practices.
  
• Troubleshoot production issues and provide timely resolution of data platform incidents.
  
• Contribute to technical documentation, knowledge transfer, and operational handover activities.
  
• Ensure adherence to security, governance, compliance, and enterprise data standards.