Cloud Data Engineer (m/w/d) – Azure Databricks

Job Description

Within BayWa r.e.’s Operations Business, we maintain and operate an asset fleet of more than 10GW of Wind & PV installations globally. We service these assets with 800+ technical and commercial colleagues, organized in local country organizations in our different markets. Our focus is to optimize the performance and longevity of each asset while ensuring the safety of our people, for both our internal and external customers.

Within Operations Business Segment, BayWa r.e. Data Services GmbH is expanding its offerings in the fields of data acquisition, SCADA services, 24/7 monitoring and data utilization as a foundation for safe operations and optimized asset performance, as well as for providing employees and customers with the best data base for adopting digital solutions.

About the role

As Cloud Data Engineer (m/f/d) – Azure Databricks you will be contributing to operate, optimize, and further development our Lakehouse data platform for operational data of renewable energy powerplants. The ideal candidate is passionate about large data processing with strong focus on quality, time series, cloud and modern data platform architecture.

This position can be located in one of our German offices in Munich, Mainz, Hamburg or Freiburg.

What You Will Do

• You will be responsible for designing, developing and maintaining scalable solar and wind park data pipelines in our Microsoft Azure Cloud and Databricks data platform.
• You will develop and maintain robust data model providing data to machine learning and data analytics experts based on our Lake House architecture.
• You will oversee unifying batch and streaming data pipeline ensuring data consistency across various source and target systems.
• You will implement data quality framework, validation processes, test suites and monitoring solution to maintain high quality standard across code, data and workflows.
• You will leverage a full suite of tool and platform including Azure, Databricks, Delta Lake, DevOps with a strong focus on time series data.

What you’ll need to succeed

• You have around 5 years of professional experience in data engineering, with a strong foundation in data architecture, cloud data platforms, and specifically Azure and Databricks.
• You’ve a strong track record on building and maintaining large datasets, code first data pipelines, data models and multi-layers data infrastructure.
• You’ve proven expertise working with the Lake House architecture, the Delta format and related big data analytics format.
• You are proficient in Python, SQL and Spark.
• You have a strong expertise on software engineering best practices, DevOps operations, CI/CD and test-driving development.
• You have good communication skills in English.

What’s in it for you?

• Being part of a highly motivated ever-growing and multicultural team that drives forward the energy revolution.
• Innovative and fast-changing market and working environment.
• Mobile working.
• Flexible working hours.
• Modern office atmosphere.
• Company restaurant / Barista bar.
• Discounted Conditions Fitness.
• Discounted Online Shopping.

You matter to us

At BayWa r.e. we are fully committed to fostering equity and inclusion and embrace diversity, of all our people, both present and those yet to join us! We welcome applications from everyone, irrespective of gender, gender identity, national origin, religion, social-, cultural-, or ethnic-background, sex, sexual orientation, age, non-disqualifying physical or mental disability, pregnancy, veteran status, or any other status protected by appropriate law. All hiring decisions are based on business needs, job requirements and qualifications. We are dedicated to being an equal opportunity employer for all by providing an inclusive environment free of discrimination or harassment.

Your Contact

Yasmine Haroun