Data Engineer - Azure Databricks

_{03rd August, 2026}

Pearl Recruitment Group is seeking an experienced Data Engineer to support a major, nationally significant program in Western Australia.

This position will help establish the core architecture for a new enterprise data platform, bringing together complex engineering, operational, supply-chain and corporate data within a Microsoft Azure environment.

You will join the program during an important foundational phase, giving you the opportunity to influence how the platform is designed, governed and scaled.

The Role

Working across a complex enterprise environment, your responsibilities will include:

• Designing and maintaining a scalable Medallion Lakehouse architecture
• Developing automated data pipelines using Azure Databricks
• Building robust ETL and ELT processes across multiple enterprise systems
• Establishing enterprise data cataloguing, modelling and governance capabilities
• Integrating engineering, operational, ERP, asset-management and corporate data
• Implementing data lineage, security and role-based access controls
• Supporting data quality, integrity and regulatory compliance requirements
• Monitoring and optimising Databricks performance, reliability and cost efficiency
• Working with technical and non-technical stakeholders to translate business requirements into practical data solutions
• Producing clear technical documentation and contributing to platform standards

About You

To be successful, you will ideally bring:

• Strong commercial experience with PySpark or Spark SQL
• Hands-on experience working within Azure Databricks
• Experience designing and building scalable data pipelines
• Strong knowledge of ETL/ELT, data integration and data-quality practices
• Exposure to enterprise data catalogues and governance tools such as Unity Catalog, Microsoft Purview or Collibra
• Experience designing complex enterprise data models
• An understanding of data lineage, security and access-control principles
• Strong stakeholder engagement and problem-solving skills
• The ability to work effectively within a complex, evolving program environment

Experience gained within Defence, manufacturing, engineering, resources, infrastructure or another complex asset-intensive industry would be highly regarded.

Previous exposure to secure or regulated data environments will also be valuable, although it is not essential for the right candidate.

Security Requirements

Candidates must be Australian citizens and eligible to obtain and maintain an AGSVA Baseline or NV1 security clearance.

What’s in It for You?

• $100 per hour
• Initial three-month contract with the possibility of extension or ongoing work
• Opportunity to contribute to a significant, long-term Australian program
• Technically challenging work across Azure, Databricks and enterprise data architecture
• Exposure to complex engineering and operational data environments
• The opportunity to help shape a foundational enterprise data platform from an early stage
• Collaborative work involving both technical specialists and senior business stakeholders

Apply Now

If you are an experienced Data Engineer with strong Azure Databricks, PySpark or Spark SQL capability, we would be interested in hearing from you.

Apply through SEEK or contact Alec on 0401 018 651 for a chat about the role.