Senior Data Engineer (AWS Databricks)

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking value and fostering innovation for its clients by harnessing world-class people and data-driven strategy. We believe that the power of people and AI can have a meaningful impact on your world, creating more fulfilling work and projects for our people and clients. For more information, visit www.blend360.com  

We are seeking a AI Engineer to contribute to our next level of growth and expansion.

We are seeking a Data Engineer to join our team, focused on data platform integration and pipeline engineering on Databricks. This role will work closely with AI Engineers to enable model development and deployment by building secure, reliable, and scalable data pipelines and integrations.

The role is not focused on data transformation or analytics modelling. Instead, it concentrates on ingestion, orchestration, connectivity, and operational pipelines that support AI and advanced analytics workloads within Databricks environments deployed into client-managed accounts.

 

Experience Required: 5+ yrs

 

Key Responsibilities

Databricks-Centric Data Engineering

• Build and maintain data pipelines that ingest data into Databricks on AWS.
• Configure and manage Databricks jobs and workflows to support AI workloads.
• Integrate Databricks with upstream source systems and downstream AI services.
• Ensure data is accessible and performant for AI training and inference use cases.

Pipeline Engineering (Non-Transformational)

• Design and implement pipelines focused on:
• Data ingestion
• Data movement
• Orchestration and scheduling
• Develop pipelines using Python and Databricks-native tooling (e.g. Databricks Jobs, workflows).
• Ensure pipelines are production-ready with monitoring, logging, and alerting.

AWS Environment Integration

• Work within client-owned AWS environments, collaborating with DevOps engineers on infrastructure provisioning.
• Integrate Databricks pipelines with cloud services such as:
• Ensure pipelines align with client security and governance requirements.

Security, Governance & Compliance

• Build and operate pipelines that meet SOC 1 compliance requirements, including:
• Access controls and permissions
• Audit logging and traceability
• Controlled deployment processes
• Support data governance standards within Databricks environments.

Delivery & Operations

• Deploy pipeline code via GitHub-based CI/CD pipelines.
• Support operational monitoring and incident response for data pipelines.
• Document pipeline designs, dependencies, and operational processes.

Core Technical Skills

• Strong experience as a Data Engineer, with a focus on Databricks-based platforms.
• Hands-on experience with Databricks on AWS, including:
• Jobs and workflows
• Cluster configuration (user-level understanding)
• Proficiency in Python for pipeline development.
• Experience using GitHub and CI/CD workflows.

Engineering & Delivery

• Experience building production-grade ingestion and orchestration pipelines.
• Ability to work effectively in client-facing delivery environments.
• Strong documentation and collaboration skills.

Nice to Have

• Experience with Databricks Unity Catalog.
• Exposure to event-driven or streaming data architectures.
• Familiarity with MLOps concepts and AI lifecycle support.