Senior Data Engineer - Multi-Cloud

UBDS Group is seeking an experienced Senior Data Engineer to design, build, and optimise scalable data platforms across complex transformation programmes. This role requires deep expertise in cloud-based data engineering, architecture, and DevOps practices, with the ability to work across both AWS and Azure environments.

You will play a key role in enabling high-quality data integration, supporting advanced analytics use cases, and delivering robust, reusable data solutions aligned to client strategies.

Key Responsibilities

• Design and develop scalable, automated data pipelines using cloud-native technologies (e.g. Spark), while integrating with on-premise data sources where required
• Implement and manage event-driven data architectures, optimising pipeline performance, parallelisation, and early risk detection (e.g. fraud use cases)
• Build and maintain data access layers and ETL frameworks, enabling both self-service and ad-hoc onboarding of data assets
• Develop reusable data engineering patterns and tooling to improve efficiency, scalability, and consistency across programmes
• Apply DevOps and software engineering best practices, including Infrastructure as Code (IaC) and CI/CD pipelines
• Collaborate with cross-functional teams to ensure high-quality data integration, governance, and performance
• Work across hybrid cloud environments (AWS and Azure), ensuring secure, resilient, and scalable solutions
• Support deployment and optimisation of data solutions leveraging services such as:
• AWS: EC2, EMR, S3, Lambda
• Azure: Blob Storage and related services

• Ensure alignment with client data strategies, governance frameworks, and security requirements
• Contribute to knowledge sharing and capability uplift within both UBDS and client teams

Skills & Experience

Essential

• Strong experience in data engineering and data architecture on complex programmes
• Proven ability to build and manage automated data pipelines using technologies such as Spark
• Experience implementing event-driven architectures and optimising data pipeline performance
• Strong knowledge of ETL processes and data integration patterns, including self-service data onboarding
• Hands-on experience with AWS and Azure cloud platforms (both required)
• Experience using key services such as:
• AWS: EC2, EMR, S3, Lambda
• Azure: Blob Storage and associated services

• Understanding of cloud networking concepts within AWS and Azure environments
• Experience with Terraform (IaC) and GitLab (CI/CD pipelines)
• Strong grounding in DevOps and modern software engineering practices

Desirable

• Experience working in government or regulated environments
• Familiarity with large-scale data platforms and data access layers
• Experience supporting fraud detection or risk analytics use cases
• Knowledge of data governance, security, and compliance frameworks

Security Requirements

• This role requires Security Clearance (SC).
• Candidates must hold active SC clearance or be eligible and willing to undergo clearance.

Engineering Practices

• Strong focus on automation, scalability, and reusability
• Advocate of DevOps culture, including CI/CD and Infrastructure as Code
• Experience working in agile, collaborative environments
• Commitment to building robust, production-grade data solutions

Personal Attributes

• Strong problem-solving and analytical skills
• Proactive and delivery-focused mindset
• Comfortable working in complex, fast-paced environments
• Excellent communication and stakeholder collaboration skills

Benefits:
Why people choose to grow their careers at UBDS Group

Professionals choose to grow their careers at UBDS Group for its reputation as a dynamic and forward-thinking organisation that is deeply committed to both innovation and employee development. At UBDS Group, employees are given unique opportunities to work on cutting-edge projects across a diverse range of industries, exposing them to new challenges and learning opportunities that are pivotal for professional growth. The Group’s culture emphasises continuous improvement, offering ample training programs, mentorship, and the chance to gain certifications that enhance their skills and marketability.

UBDS Group fosters a collaborative environment where creativity and innovation are encouraged, allowing employees to contribute ideas and solutions that have a tangible impact on the company and its clients. This combination of professional development, a culture of innovation, and the opportunity to make meaningful contributions makes UBDS Group an attractive place for those looking to advance their careers and be at the forefront of technological and operational excellence.

Employee Benefits

• Training – All team members are offered a number of options in terms of personal development, whether it is technical led, business acumen or methodologies. We want you to grow with us and to help us achieve more
• Private medical cover for you and your spouse/partner, offered via Vitality
• Discretionary bonus based on a blend of personal and company performance
• Holiday – You will receive 25 Days holiday, plus 1 day for Birthday and 1 day for your work anniversary in addition to UK bank holidays
• Electric Vehicle leasing with salary sacrifice
• Contributed Pension Scheme
• Death in service cover

About UBDS Group

At UBDS Group our mission is to support entrepreneurs who are setting new standards with technology solutions across cloud services, cybersecurity, data and AI, ensuring that every investment advances our commitment to innovation, making a difference, and creating impactful solutions for organisations and society.

Equal Opportunities

We are an equal opportunities employer and do not discriminate on the grounds of gender, sexual orientation, marital or civil partner status, pregnancy or maternity, gender reassignment, race, colour, nationality, ethnic or national origin, religion or belief, disability or age.