Senior Data Engineer - AI-Accelerated Data Architecture & Platforms

Whiteshield is a global policy advisory and AI economics firm known for our ability to respond to global challenges rapidly and incisively. We use the most advanced tools and technology and combine them with our team of leading international experts to engage decision-makers in tackling society's most significant challenges. We are recognized for our rapid decision support, innovation, data science algorithms and deep policy expertise. We are specialists in "connecting the dots" between policy, business and technology. 

Whiteshield X is Whiteshield’s product business, hosting our PaaS and SaaS product offering. Partnered with Google Cloud, we bring advanced solutions to governments around the world to drastically improve citizen outcomes in areas such as employment, trade, and education. We develop a host of different SaaS modules from job matching to managing scholarship programs, and from delivering financial support to growth support for SMEs. These modules support governments around the world with delivering G2C and G2B services efficiently and significantly improve policy outcomes.

We are seeking a Senior Data Engineer who combines deep technical expertise with architectural vision to build the next generation of AI-accelerated data platforms.

You will play a pivotal role in defining and implementing the organization’s data architecture and platform strategy — ensuring that data is reliable, well-structured, and easily consumable across systems and applications. This is both a strategic and hands-on position: you’ll design scalable architectures, build data pipelines, and collaborate with application and product teams to embed data excellence into every part of the technology stack.

If you’re passionate about data craftsmanship, AI-driven automation, and modern cloud-native ecosystems, this is an opportunity to make a large-scale impact.

What You'll Do

• Data Architecture & Strategy

• Define and maintain the enterprise data architecture blueprint, spanning ingestion, lake, warehouse, and semantic layers.

• Collaborate closely with application and platform teams to design robust data strategies — ensuring alignment between application design, API contracts, and long-term data scalability.

• Develop integration patterns and governance models that support cross-platform interoperability, data lineage, and secure data sharing.

• Partner with cloud architects to ensure performance, scalability, and compliance across Azure environments.

• Data Engineering & Orchestration

• Build and optimize end-to-end data pipelines using Apache Airflow, Azure Fabric, Azure Data Factory, and SSIS — integrating diverse data sources (transactional, streaming, and analytical).

• Implement AI-accelerated data processing for intelligent schema mapping, deduplication, and performance optimization.

• Develop and maintain ETL/ELT frameworks using SQL and Python (Pandas, PySpark) for high-volume, low-latency data movement.

• Data Modeling & Warehousing

• Design semantic data models, dimensional schemas, and data marts using SSAS, PostgreSQL, MSSQL, and Azure Synapse/Fabric.

• Collaborate with BI and analytics teams to support dashboards, reports, and forecasting tools with clean, contextualized, and well-documented data models.

• Performance, Quality & Governance

• Define standards for data quality, lineage, observability, and access control.

• Implement automated validation, monitoring, and reconciliation mechanisms.

• Work with governance and security teams to ensure compliance with national and organizational data frameworks.

• Collaboration & Enablement

• Partner with application engineers, data scientists, and business analysts to deliver AI-ready, high-quality data products.

• Act as a bridge between application development and data architecture, ensuring consistent data design across services.

• Mentor junior data engineers and champion best practices in modern data engineering and DevOps for data.

Requirements:
Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related discipline.

• 6+ years of experience in data engineering and architecture, preferably in complex or government-scale environments.

Expertise in:

• Orchestration: Apache Airflow, Azure Data Factory, or equivalent.

• ETL / BI Stack: SSIS, SSAS, dbt, Power BI semantic models.

• Databases: MSSQL, PostgreSQL, Azure Synapse, Azure Fabric Data Lake.

• Cloud Platforms: Deep knowledge of Azure (Fabric, Synapse, Blob, Data Lake, Functions, Logic Apps). Google Cloud is a plus.

• Languages: SQL and Python (Pandas, PySpark); experience with CI/CD pipelines and GitOps practices.

• Demonstrated experience designing AI-ready architectures and leveraging machine learning for data enrichment and automation.

• Strong understanding of data governance, interoperability standards, and scalable architecture design patterns.

Benefits:
WHY CHOOSE US |

Our Commitment to Equal Opportunities: At Whiteshield, we are committed to providing equal opportunities in employment regardless of individual characteristics. We recognize that our employees feel appreciated when their thoughts and values are respected and considered. We’re committed to maintaining and driving an inclusive culture and workplace where all talents are nurtured and feel empowered to contribute.

Celebrating Achievements: At Whiteshield, we seek to reward those who demonstrate exceptional individual achievements above and beyond expectations in the pursuit of continuously seeking excellence for our clients. When individuals excel, we show our appreciation and celebrate achievements by generously recognizing and supporting the career progression of those who make unique contributions to the firm.

Empowering Our People: At Whiteshield, we recognize that our employees are the core to our success and believe that when our exceptional talent comes together, and wellbeing is a priority – innovation thrives. We are dedicated to promoting and encouraging a healthy work-life balance and adopting a genuinely flexible working model to empower our employees to pursue a balance that suits their personal needs.