Data engineer with Fabric, Lakehouse, Azure and python, notebooks needed!

Project description

The client is in the process of implementing a lakehouse architecture in Microsoft Fabric and is finalising a roadmap for the coming years to be presented to the leadership team in early November. The roadmap initially extends to 2027, with testing and rollout to multiple countries.

Ongoing and planned activities:

• Onboarding data from multiple sources onto the platform
• Delivering several AI artefacts within the setup
• Building capabilities for end users
• Two AI pilots planned for next year – including Copilot functionality towards end users in the app
• Exploration of machine learning and predictions (revenue predictions, operational predictions, etc.)
• Scoping exercise in November to identify the most important business questions
• AI integrated into the app, used as a communication tool for end users and unit managers (operational tool)
• Claude is connected to the platform

Work is under way to provide unit managers in the organisation with better tools to understand and use data.

Team

Internal data team consisting of:

• Data engineer
• One person responsible for Power BI work
• One person working at the intersection of data engineering and Power BI

Distributed team

Responsibilities

• Work closely with the data engineer and the rest of the internal data team
• Contribute to onboarding data onto the Fabric Lakehouse platform
• Deliver AI artefacts and build capabilities for end users
• Participate in pilots related to AI towards end users and machine learning predictions
• Communicate with the business side to understand needs and translate these into technical solutions

Requirements

Must have:

• Microsoft Fabric
• Lakehouse architecture
• Python
• Python Notebooks
• Data engineering
• Data science
• AI/ML
• Collaboration with cross-functional teams
• Ability to take ownership

Nice to have:

• Experience with Azure Synapse
• Power BI
• Machine learning predictions
• Copilot/AI integration
• Communication with the business side
• International team experience

Personal attributes

• Takes ownership – not just doing what one is told
• Thinks long-term and focuses on what is sustainable and low-maintenance over time
• Collaborates closely with the team – not a solo operator
• Able to engage with the business side and understand what they want
• Highly dedicated

Selling points

• Work in an international team with high dedication and ownership
• Exciting technology stack: Microsoft Fabric, Python Notebooks, Lakehouse, AI/ML
• Direct impact on product development in the app used by end users and unit managers
• A friendly, distributed team with a high degree of remote working

Next steps

The client is in the initial phase and is clarifying the right path forward, including budget. A decision on whether to bring in a consultant is expected around mid-November.