Data Engineer (PySpark)

GSSTech Group

Dubai, Dubai, United Arab Emirates permanent

Posted: February 19, 2026

Job Description

We are seeking a highly skilled Data Engineer with strong expertise in PySpark and the Cloudera Data Platform (CDP). The ideal candidate will design, develop, and maintain scalable data pipelines while ensuring high data quality, performance, and availability across the organisation.

This role requires hands-on experience in big data ecosystems, cloud-native technologies, and advanced data processing frameworks. You will collaborate with cross-functional teams to build reliable and high-performance data solutions that drive business insights.

Key Responsibilities

1. Data Pipeline Development

• Design, develop, and maintain scalable ETL/ELT pipelines using PySpark on CDP
• Ensure data integrity, reliability, and performance optimisation

2. Data Ingestion

• Develop ingestion frameworks to collect data from relational databases, APIs, streaming sources, and file systems
• Load structured and unstructured data into Data Lake/Data Warehouse environments

3. Data Transformation & Processing

• Process, cleanse, and transform large-scale datasets using PySpark
• Build reusable data processing components

4. Performance Optimisation

• Tune Spark jobs and Cloudera components for optimal performance
• Optimise memory, partitioning, and execution plans
• Reduce ETL runtime and improve cluster efficiency

5. Data Quality & Validation

• Implement data validation checks and monitoring mechanisms
• Ensure end-to-end data quality and governance standards

6. Automation & Orchestration

• Automate workflows using tools such as Apache Oozie, Apache Airflow, or similar orchestration frameworks
• Maintain CI/CD integration for data pipelines

7. Monitoring & Support

• Monitor pipeline health and troubleshoot failures
• Provide production support and continuous improvements

Required Skills & Qualifications

• 5+ years of experience in Data Engineering
• Strong hands-on experience in PySpark
• Experience working on Cloudera Data Platform (CDP)
• Strong knowledge of Hadoop ecosystem (HDFS, Hive, Impala, YARN)
• Proficiency in SQL and data modelling concepts
• Experience with workflow orchestration tools (Airflow, Oozie, etc.)
• Good understanding of data warehousing concepts
• Experience with performance tuning and optimisation

Good to Have

• Experience with cloud platforms (AWS, Azure, GCP)
• Knowledge of streaming tools (Kafka, Spark Streaming)
• Exposure to DevOps practices and CI/CD pipelines
• Banking/Financial Services domain experience

Why Apply Through MisuJob?

AI-Powered Job Matching: MisuJob uses advanced artificial intelligence to analyze your skills, experience, and career goals. Our matching algorithm compares your profile against thousands of job requirements to find positions where you have the highest chance of success. This saves you hours of manual job searching and ensures you only see relevant opportunities.

One-Click Applications: Once you create your profile, applying to jobs is effortless. Your resume and cover letter are automatically tailored to highlight the most relevant experience for each position. You can apply to multiple jobs in minutes, not hours.

Career Intelligence: Beyond job matching, MisuJob provides valuable career insights. See how your skills compare to market demands, identify skill gaps to address, and understand salary benchmarks for your experience level. Make data-driven decisions about your career path.

Frequently Asked Questions

How do I apply for this position?

Click the "Register to Apply" button above to create a free MisuJob account. Once registered, you can apply with one click and track your application status in your dashboard.

Is MisuJob free for job seekers?

Yes, MisuJob is completely free for job seekers. Create your profile, get matched with jobs, and apply without any cost. We help you find your dream job without any hidden fees.

How does AI matching work?

Our AI analyzes your resume, skills, and experience to understand your professional profile. It then compares this against job requirements using natural language processing to calculate a match percentage. Higher matches mean better fit for the role.

Can I apply to jobs in other countries?

Absolutely. MisuJob features jobs from companies worldwide, including remote positions. Filter by location or look for remote opportunities to find jobs that match your preferences.

Ready to Apply?

Join thousands of job seekers using MisuJob's AI to find and apply to their dream jobs automatically.

Interested in this position?

Required Skills