Direct message the job poster from Astek Middle East
Overview
We are seeking an experienced Data Engineer with strong expertise in the Cloudera ecosystem to design, build, and optimize scalable data pipelines and architectures. You will be responsible for data integration, ETL / ELT development, ensuring data quality and performance, and supporting advanced analytics and business intelligence needs.
Responsibilities
- Design, build, and optimize scalable data pipelines and architectures.
- Develop ETL / ELT processes to integrate structured and unstructured data.
- Collaborate with analysts, data scientists, and business stakeholders to deliver reliable data solutions.
- Ensure data quality, consistency, security, and compliance across platforms.
- Optimize database queries and big data processing for performance and scalability.
- Monitor and support daily data pipelines, ETL jobs, and workflows.
- Troubleshoot failures in data jobs and coordinate with engineering teams for resolution.
- Implement automation and monitoring tools to improve operational efficiency.
- Document processes, incidents, and solutions for future reference.
Required Qualifications
Bachelor’s degree in Computer Science, Engineering, or related field .6+ years of experience in data engineering.Proven experience with the Cloudera ecosystem (Spark, Hive, Kafka, HDFS).Strong proficiency in SQL, Python, and data modeling .Solid understanding of data integration, warehousing, and architecture design .Preferred Qualifications
Experience with CI / CD pipelines and version control tools (e.g., Git).Familiarity with cloud platforms (AWS, Azure, or GCP).Strong problem-solving and analytical skills.Ability to design efficient and scalable data solutions.Strong collaboration and communication with diverse teams.High attention to detail with a focus on data quality and reliability .Seniority level
Mid-Senior levelEmployment type
Full-timeJob function
Information Technology#J-18808-Ljbffr