AWS DATA Engineer

This role is for one of the Weekday's clients

Salary range: Rs 2600000 - Rs 4300000 (ie INR 26-43 LPA)

Min Experience: 6 years

Location: Hyderabad

JobType: full-time

We are seeking a highly skilled and experienced AWS Data Engineer to design, build, and optimize scalable data platforms in a cloud-first environment. This role is ideal for professionals with 6–14 years of experience who possess deep expertise in AWS services, strong programming skills in Python, hands-on experience with PySpark, and working knowledge of modern table formats such as Apache Iceberg. You will play a critical role in enabling data-driven decision-making by developing reliable, high-performance data pipelines and analytics-ready architectures.

Requirements:
Key Responsibilities

• Design, develop, and maintain robust data pipelines on AWS to process structured and unstructured data at scale.
• Build scalable ETL/ELT workflows using Python and PySpark to transform raw data into analytics-ready datasets.
• Architect and manage data lakes and lakehouse architectures leveraging Amazon S3 and Apache Iceberg for optimized storage and transactional consistency.
• Implement data ingestion frameworks using services such as AWS Glue, AWS Lambda, Amazon EMR, and Amazon Kinesis.
• Optimize performance and cost efficiency across data workloads running on AWS.
• Develop and maintain data models that support BI tools, advanced analytics, and machine learning initiatives.
• Ensure data quality, governance, and security by implementing validation checks, encryption, and access controls.
• Collaborate closely with Data Scientists, Analytics teams, DevOps, and business stakeholders to translate requirements into scalable technical solutions.
• Automate deployment and monitoring processes using CI/CD pipelines and infrastructure-as-code tools.
• Troubleshoot and resolve performance bottlenecks, data inconsistencies, and system issues in distributed environments.

Required Skills & Expertise

AWS Data Engineering:
Strong hands-on experience with core AWS data services including Amazon S3, AWS Glue, Amazon EMR, Amazon Redshift, AWS Lambda, AWS Step Functions, and Amazon Athena. Proven ability to architect secure, scalable, and cost-effective cloud data solutions.

Python:
Advanced proficiency in Python for data engineering tasks, including writing modular, maintainable, and high-performance code. Experience with data processing libraries, API integrations, automation scripts, and workflow orchestration.

PySpark:
Extensive experience building distributed data processing jobs using PySpark. Ability to optimize Spark jobs, handle large-scale datasets, and tune performance across cluster environments.

Apache Iceberg:
Hands-on experience implementing Apache Iceberg in data lake environments. Strong understanding of schema evolution, partitioning strategies, time travel, and ACID transaction support in modern lakehouse architectures.

Qualifications

• 6–14 years of overall IT experience with a strong focus on data engineering.
• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• Proven experience designing and maintaining large-scale data platforms in production environments.
• Strong understanding of distributed systems, data modeling, and cloud-native architecture.
• Experience with version control systems and CI/CD best practices.
• Excellent problem-solving, communication, and stakeholder management skills.

Preferred Qualifications

• Experience with data governance frameworks and data catalog tools.
• Familiarity with real-time streaming architectures.
• AWS certifications (e.g., AWS Certified Data Analytics or Solutions Architect).