Lead Data Engineering Architect
Westbury Partners Sydney, AustraliaLead Data Engineering Architect
Lead the design and delivery of scalable data platforms and pipelines, combining technical expertise, architecture, mentorship, and strategic leadership to enable trusted, high-performing, data-driven decisions.
What You'll Do:
Shape the future of data engineering by leading the architecture, development, and delivery of scalable data platforms, pipelines, and analytical solutions. You’ll turn complex data challenges into reliable, high-performance solutions that create meaningful business impact.
Your responsibilities will include:
- Lead the design and development of scalable ETL/ELT pipelines and modern data platforms.
- Define engineering standards, architecture patterns, development methodologies, and best practices.
- Build and optimise data warehouses, data lakes, and lakehouse solutions.
- Champion data quality, security, governance, availability, and performance.
- Lead architecture reviews, technical discussions, code reviews, and solution design sessions.
- Mentor data engineers and promote strong engineering practices across the team.
- Establish effective monitoring, testing, documentation, CI/CD, and deployment processes.
- Troubleshoot complex pipeline, performance, infrastructure, and data issues.
- Evaluate emerging technologies and recommend solutions that balance innovation, scalability, and cost.
- Partner with Data Scientists, BI Developers, Analysts, Software Engineers, Product Managers, and business stakeholders.
- Drive continuous improvement across data architecture, engineering processes, productivity, and technical debt.
- Support cloud-based initiatives while ensuring robust, scalable, and cost-effective solutions.
Why Join Us:
This is an opportunity to combine hands-on engineering with genuine technical leadership. You’ll have the influence to shape data architecture, establish engineering standards, mentor talented professionals, and deliver platforms that directly support business growth and smarter decision-making.
About You:
- You’re an experienced data engineering professional with7+ years of relevant experience and at least2 years leading or mentoring engineering teams . You bring strong SQL and programming skills, extensive ETL/ELT experience, and a solid understanding of data modelling, warehousing, distributed systems, and scalable data processing.
- You’re comfortable working with cloud platforms and modern technologies such as Spark, Kafka, Airflow, Databricks, Snowflake, BigQuery, or equivalent solutions. You understand software engineering practices including version control, CI/CD, automated testing, and deployment.
- You’re also a confident technical leader who can communicate complex ideas clearly, collaborate across disciplines, solve challenging problems, and make pragmatic architectural decisions.
#DataEngineering #DataEngineer #DataEngineeringLead #DataArchitecture #DataPipelines #ETL #ELT #CloudComputing #DataWarehouse #DataLake #DataLakehouse #ApacheSpark #Databricks #DataGovernance #TechnicalLeadership