Databricks Developer/Data Engineer

About Us: Renaissance Info Systems is a technology and digital recruitment agency, connecting contract and permanent professionals with clients across Asia-Pacific. We aim to differentiate ourselves through our level of responsiveness, and our understanding that comes from being an IT recruitment agency from the IT Industry. Our recruiters balance sophisticated and simple inter-personal techniques to assure a strong candidate network. Know More: http://www.reninfo.com.au
Databricks DeveloperLocation: MelbourneWork Model: HybridEmployment Type: ContractExperience: 5+ years in Data Engineering, with strong hands-on Databricks experienceAbout the RoleWe are seeking an experienced Databricks Developer to design, develop and optimise scalable data engineering solutions using Databricks, PySpark, Python and SQL. The successful candidate will work closely with Data Engineers, Architects, Analysts and business stakeholders to deliver reliable, high-performance data pipelines and modern lakehouse solutions.Key ResponsibilitiesDesign, develop and maintain scalable ETL/ELT data pipelines using Databricks and PySpark.Develop complex data transformations using Python, PySpark, Spark SQL and SQL.Build and manage Delta Lake tables and implement Bronze, Silver and Gold data layers.Develop batch and, where required, streaming data pipelines using Spark/Structured Streaming.Integrate Databricks with Azure Data Factory, ADLS Gen2, Azure Synapse and other cloud data services.Perform Spark job and query performance tuning, including partitioning, caching and optimisation.Essential Skills & Experience5+ years of experience in Data Engineering / Data Development.Strong hands-on experience with Databricks.Advanced PySpark and Python development skills.Strong SQL / Spark SQL skills, including complex joins, aggregations and window functions.Strong experience with Delta Lake and Lakehouse architecture.Experience with Azure Data Factory (ADF) and Azure Data Lake Storage Gen2 (ADLS).Strong understanding of ETL/ELT, data warehousing and data modelling.Experience developing scalable batch data pipelines.Knowledge of Medallion Architecture and incremental/CDC processing.Experience with Git and CI/CD pipelines.Strong troubleshooting and performance optimisation skills.Experience working in Agile / Scrum enterprise environments.

Regards,Reshu SethRecruitment ConsultantRenaissance InfoSystemM: +61 478 487 026E: reshu@reninfo.com.auW: http://www.reninfo.com.au