Data Engineer

Responsibilities

• Design, develop, and maintain scalable data pipelines for the efficient collection, storage, and processing of large datasets.
• Develops and design best practices around data collection and maintain orchestration of data flows, for batch, near-real time data.
• Develop and design best practices of CDC and SCD for existing systems to maintain data consistency
• Participates in evaluation and deployment of big data tools, relevant to the development.
• Ability to identify different platforms, techniques, and methodology used in big data platforms, and demonstrate for various system in the organization.
• Optimize and tune data systems for performance and reliability.
• Develop and implement data models and schemas to support business intelligence and analytics needs.
• Monitor and troubleshoot data pipelines to ensure smooth and efficient operation.
• Stay up-to-date with the latest advancements in data engineering and incorporate them into our data infrastructure.
• Document data engineering architectures and follow best practices.

Qualifications

#J-18808-Ljbffr