Discover your dream Career
For Recruiters  /  人才招聘

Lead Data Engineering Architect

Westbury Partners Sydney, Australia
Posted 13 hours ago Permanent A$400k - A$700k

Lead Data Engineering Architect

Westbury Partners Sydney, Australia

Lead the design and delivery of scalable data platforms and pipelines, combining technical expertise, architecture, mentorship, and strategic leadership to enable trusted, high-performing, data-driven decisions.

What You'll Do:

Shape the future of data engineering by leading the architecture, development, and delivery of scalable data platforms, pipelines, and analytical solutions. You’ll turn complex data challenges into reliable, high-performance solutions that create meaningful business impact.

Your responsibilities will include:

  1. Lead the design and development of scalable ETL/ELT pipelines and modern data platforms.
  2. Define engineering standards, architecture patterns, development methodologies, and best practices.
  3. Build and optimise data warehouses, data lakes, and lakehouse solutions.
  4. Champion data quality, security, governance, availability, and performance.
  5. Lead architecture reviews, technical discussions, code reviews, and solution design sessions.
  6. Mentor data engineers and promote strong engineering practices across the team.
  7. Establish effective monitoring, testing, documentation, CI/CD, and deployment processes.
  8. Troubleshoot complex pipeline, performance, infrastructure, and data issues.
  9. Evaluate emerging technologies and recommend solutions that balance innovation, scalability, and cost.
  10. Partner with Data Scientists, BI Developers, Analysts, Software Engineers, Product Managers, and business stakeholders.
  11. Drive continuous improvement across data architecture, engineering processes, productivity, and technical debt.
  12. Support cloud-based initiatives while ensuring robust, scalable, and cost-effective solutions.

Why Join Us:

This is an opportunity to combine hands-on engineering with genuine technical leadership. You’ll have the influence to shape data architecture, establish engineering standards, mentor talented professionals, and deliver platforms that directly support business growth and smarter decision-making.

About You:

  1. You’re an experienced data engineering professional with7+ years of relevant experience and at least2 years leading or mentoring engineering teams . You bring strong SQL and programming skills, extensive ETL/ELT experience, and a solid understanding of data modelling, warehousing, distributed systems, and scalable data processing.
  2. You’re comfortable working with cloud platforms and modern technologies such as Spark, Kafka, Airflow, Databricks, Snowflake, BigQuery, or equivalent solutions. You understand software engineering practices including version control, CI/CD, automated testing, and deployment.
  3. You’re also a confident technical leader who can communicate complex ideas clearly, collaborate across disciplines, solve challenging problems, and make pragmatic architectural decisions.

#DataEngineering #DataEngineer #DataEngineeringLead #DataArchitecture #DataPipelines #ETL #ELT #CloudComputing #DataWarehouse #DataLake #DataLakehouse #ApacheSpark #Databricks #DataGovernance #TechnicalLeadership

Job ID  8925
More Jobs From Westbury Partners
Westbury Partners
Research Engineer
Westbury Partners
Sydney, Australia
3 days ago Full time A$400k - A$700k
Westbury Partners
Python Backend Engineer
Westbury Partners
Sydney, Australia
4 days ago Full time S$400k - S$700k
Westbury Partners
Trading Tools & Data Engineering Specialist
Westbury Partners
Hong Kong
18 hours ago Full time HK$2m - HK$3m
Westbury Partners
Junior Engineering Operations & Projects Engineer
Westbury Partners
Sydney, Australia
13 hours ago Full time A$300k - A$600k
Westbury Partners
Blockchain Infrastructure & Production Engineer
Westbury Partners
Sydney, Australia
14 hours ago Full time A$400k - A$700k
Westbury Partners
Trading Systems Software Engineer
Westbury Partners
Sydney, Australia
3 days ago Full time A$400k - A$700k
Westbury Partners
Secure Platform Reliability Engineer
Westbury Partners
Sydney, Australia
3 days ago Full time A$400k - A$700k
Westbury Partners
Linux HPC Operations Engineer
Westbury Partners
Sydney, Australia
3 days ago Full time A$400k - A$700k
Westbury Partners
Low-Latency Performance Optimization Engineer
Westbury Partners
Sydney, Australia
4 days ago Full time A$350k - A$500k
Westbury Partners
Cyber Security Operations Engineer - HFT - SYD
Westbury Partners
Sydney, Australia
11 hours ago Full time A$250k - A$600k

Boost your career

Find thousands of job opportunities by signing up to eFinancialCareers today.
More Jobs Like This
Hays
Lead Data Engineer
Hays
Sydney, Australia
Commonwealth Bank of Australia
Staff Software Engineer, Customer Channel, Data & AI
Commonwealth Bank of Australia
Sydney, Australia
Hays
Senior Data Engineer
Hays
Sydney, Australia
Commonwealth Bank of Australia
Chapter Lead, Data Science (Operations)
Commonwealth Bank of Australia
Sydney, Australia
Commonwealth Bank of Australia
Staff Data Engineer
Commonwealth Bank of Australia
Sydney, Australia