Vacant job
- Jobs
- Senior Data Engineer – Data Lake & ETL (Python, Spark, Iceberg)
Senior Data Engineer – Data Lake & ETL (Python, Spark, Iceberg)
Avaron ABStockholms län, Solna
Previous experience is desired
8 days left
to apply for the job
At Avaron, you gain the security of a permanent position combined with the variety of working on-site at different clients. We recruit specialists in everything from technology, IT, and industry to project management and business support – and regardless of the assignment, you have a consulting manager who is there for you and your development.
About the RoleYou will step into a role where data is crucial for improving healthcare, research, and data-driven working methods in a complex and regulated environment. Here, you will help finalize and further develop a Data Lake platform and research infrastructure that makes clinical data more accessible, structured, and quality-assured.
You will join an active development phase and work closely with existing architecture, in a technical environment built on metadata-driven ETL design, Apache Iceberg, Spark, Trino, PII anonymization, and OpenShift. This is a role for you who wants to combine advanced system development with a clear impact on both operations and research.
Responsibilities- You finalize and further develop a metadata-driven ETL pipeline framework for a research platform.
- You develop and maintain data pipelines, backend services, and integrations with clinical source systems and databases.
- You further develop the Data Lake architecture using Apache Iceberg, Spark, and Trino.
- You ensure correct handling, anonymization, and protection of sensitive clinical data in accordance with current regulations.
- You drive quality in delivery through test-driven development, code reviews, and automation.
- You collaborate closely with developers, product owners, and architects to create scalable and sustainable solutions on OpenShift.
- An academic degree in Computer Science, System Development, or equivalent documented experience.
- At least 5 years of experience in system development with Python as the primary language, focusing on data pipelines, backend services, and system integration.
- At least 3 years of experience in ETL/ELT development targeting SQL databases, such as MySQL and MSSQL, as well as object storage, such as AWS S3 and Ceph.
- At least 3 years of experience with event-driven architecture and async message handling using Kafka, RabbitMQ, or similar.
- At least 3 years of experience working with container platforms such as Kubernetes or OpenShift, as well as CI/CD solutions like Jenkins, Bamboo, or GitLab CI.
- At least 2 years of experience with distributed data processing using Apache Spark, including integration with Data Lake platforms.
- Documented experience with Apache Iceberg or Delta Lake as an open table format in a production environment.
- Documented experience with metadata-driven ETL design and pipeline frameworks in a production environment.
- Documented experience with PII anonymization, encryption, or hashing of sensitive clinical data according to regulatory requirements in a production environment.
- Documented experience with Trino or a similar distributed SQL query engine against a Data Lake in a production environment.
- Experience collaborating with both technical and business-oriented teams.
- You work systematically with a focus on quality assurance and risk management, have a good ability to work independently, and can handle multiple tasks in parallel.
- Fluent in Swedish and English, both spoken and written.
- Experience developing data pipelines using Data Lake technologies in publicly funded healthcare or research activities involving clinical data.
- Experience independently designing and implementing Data Lake architecture with open table formats in publicly funded healthcare or research activities involving clinical data.
- Experience using and configuring monitoring for implemented services or features in a production environment.
- Experience independently designing and implementing test and quality assurance frameworks for data pipelines in a production environment.
- Experience designing and implementing security solutions for systems handling sensitive data.
- Permanent employment at Avaron AB
- Private pension plan
- Wellness allowance of 5,000 SEK per year
We recruit continuously – please apply as soon as possible.
🖐 Was this job fit for someone?
Other jobs in the same field
Maybe it’s time to broaden the search with these available jobs
-
Up to 25% off experiences for mom – Celebrate Mother’s Day with Live it
Tue, 26 May 2026 - 12:00