Vacant job

Job Description

The client is undergoing a comprehensive digital transformation – and data is central to the journey. They are building a modern, decentralized data platform inspired by data mesh principles, where teams can publish, discover, and consume reliable data products in a secure and standardized way.

We are now looking for a Senior Data Engineer to join the client’s cross-functional Data Foundry team. The team is a hybrid team that develops shared platform capabilities – such as catalog, governance, orchestration, and SDKs – and works hands-on with real business-related use cases.
By building data products themselves, they ensure that the platform works in practice and that they create business value early.

The role is hands-on with a clear focus on building and further developing data products and platform patterns in Azure and Databricks, combined with enabling other teams when it has the greatest impact.

This is what you will work on

• Building and Leading the Way
  You will develop data products hands-on by working with real data sources, building pipelines, and publishing reusable datasets in Databricks.
  You will create reference implementations according to the client’s golden path that demonstrate how teams onboard, transform, and expose data using Unity Catalog and Databricks-native tools. You will evaluate and drive design choices and trade-offs between Spark Declarative Pipelines, Databricks Jobs, SDK-based orchestration, and various deployment methods.
  
• Enablement and Collaboration
  You will help other development teams onboard their data products to the platform and get started with the client’s golden path, including structure in Unity Catalog, metadata management, and secure access patterns. At times, you will work closely with other teams to guide them through the setup and deployment of their first data products. You will gather feedback from real usage and use it to improve architecture patterns, tooling, and documentation.
  
• Driving the Platform Forward
  You will contribute to internal platform development: SDKs, scaffolded templates, Databricks Asset, Bundles (DAB)-based workflows, and CLI tools.
  You will design patterns for automation and CI/CD, for example with GitHub Actions, to manage data pipelines and Databricks resources.
  You will build in compliance and governance by design – GDPR, the right to be forgotten (RtBF), data classification, and privacy-preserving transformations. You will continuously work to improve the developer experience based on feedback and insights from how the platform is used.

Experience Requirements

• At least 5 years of practical data engineering experience, preferably in a cloud-based environment.
• Deep experience with Databricks, including Unity Catalog, Jobs, Workflows, as well as workspace and cluster configuration.
• Comfortable designing and orchestrating data pipelines, preferably with Databricks-native tools such as Spark Declarative Pipelines. Experience with Dagster or Airflow is a plus.
• Understanding trade-offs between different orchestration and architecture patterns and can make pragmatic choices.
• Comfortable with Python, SQL, and Git-based workflows.
• Good understanding of data governance, privacy, and access control.
• Enjoy working cross-functionally, coaching others, and documenting technical solutions clearly.

Preferred Qualifications

• Experience automating Databricks environments with GitHub Actions.
• Familiarity with Backstage or other developer portals.
• Experience enabling Power BI or other analytical consumption patterns.
• Experience in internal platform development and reusable templates/patterns.

Your Application

Does this role sound interesting and suitable? If so, please apply immediately as we are interviewing continuously and the role may be filled before the application deadline.

We can only accept and process your application by having you register your CV in our portal. With respect to GDPR, we cannot accept applications via email. We warmly welcome your application! This assignment is part of Quest Consulting's staffing services.

About Us

Quest Consulting is an authorized consulting company with collective agreements, insurance, wellness programs, and occupational pensions. We specialize in IT, Engineering, HR, Administration, and Finance. Our goal is to be your personal partner, and that is why it is so important for us to work according to our core values where our keywords are to be Personal, Innovative, and Professional.