JOB SUMMARY: The Data Engineer will design, develop, and maintain data pipelines and infrastructure to support business intelligence, analytics, and machine learning initiatives. This role requires a deep understanding of data integration, data modeling, and performance optimization to enable efficient data processing and analysis. The ideal candidate will ensure data availability, reliability, and security while collaborating with cross-functional teams to drive data-driven decision-making across the organization.
RESPONSIBILITIES:
- Develop, test, and maintain scalable ETL (Extract, Transform, Load) processes to integrate data from various sources.
- Build and optimize data pipelines and architectures for both structured and unstructured data.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
- Implement and maintain data security protocols to ensure compliance with industry standards and regulations.
- Perform data quality checks and ensure consistency, accuracy, and reliability of datasets.
- Monitor and troubleshoot data pipelines, resolving any issues to maintain data flow.
- Design and implement database schemas and data storage solutions for performance optimization.
- Stay updated on emerging technologies and trends in data engineering to recommend improvements.