Senior Lead Data Engineer - R01560164

Senior Lead Data Engineer

Primary Skills:
• AKS, Event Hub, Azure DevOps, Cosmos DB, Azure Functions

Specialization:
• Azure Data Engineering Advanced: Senior Data Engineer

Job requirements:
• Key Responsibilities:- ETL Pipeline Monitoring & Support o Monitor and manage multiple ETL pipelines supporting Novacore data platforms. o Provide end-to-end troubleshooting, debugging, and resolution of pipeline failures. o Perform root cause analysis for issues and implement permanent fixes. o Proactively communicate pipeline failures, resolutions, and status updates to internal stakeholders and clients. 1. ETL Development & Architecture Design o Design and develop new ETL pipelines and data processing workflows as per business requirements. o Assist clients in defining data architecture and proposing optimal technical solutions. o Recommend and implement scalable, efficient, and cost-effective ETL and data architecture approaches. o Support enhancements and optimizations to existing pipelines and data models. 2. Production Deployment & DevOps Support o Manage code deployments to Production environments following established release processes. o Utilize Azure DevOps for source control, CI/CD pipelines, release management, and work item tracking. o Ensure deployment best practices, version control, and rollback strategies are followed. 3. Operational Support, Documentation & Collaboration o Provide ongoing operational support and maintenance for data pipelines and related systems. o Create and maintain technical documentation, including pipeline workflows, architecture diagrams, and runbooks. o Collaborate with cross-functional teams (engineering, analytics, and business stakeholders) to ensure smooth delivery and alignment with business goals. o Adhere to agreed SLAs, timelines, and quality standards.