Senior Big Data Engineer
Data Engineering•Full Time•8+ years•Kuala Lumpur, Wilayah persekutuan kuala lumpur•8 k - 8 k
Required Skills4 skills
Apache Spark
Hive
Apache Iceberg
Medallion framework
Job Description
<h3>Job Overview</h3><p>We are seeking a highly skilled and experienced Senior Big Data Engineer to join our team. As a Senior Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work closely with data scientists and other engineers to ensure the scalability, reliability, and performance of our big data platform.</p><h3>Key Responsibilities</h3><ul><li>Design and implement big data solutions using Apache Spark, Hive, and other big data technologies.</li><li>Develop and maintain data pipelines for data ingestion, processing, and storage.</li><li>Optimize big data infrastructure for performance and scalability.</li><li>Troubleshoot and resolve issues related to big data infrastructure.</li><li>Collaborate with data scientists and other engineers to develop and deploy machine learning models.</li></ul><h3>Required Skills</h3><ul><li>Apache Spark</li><li>Hive</li><li>Apache Iceberg</li><li>Medallion framework</li></ul>Job Overview
We are seeking a highly skilled and experienced Senior Big Data Engineer to join our team. As a Senior Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work closely with data scientists and other engineers to ensure the scalability, reliability, and performance of our big data platform.
Key Responsibilities
- Design and implement big data solutions using Apache Spark, Hive, and other big data technologies.
- Develop and maintain data pipelines for data ingestion, processing, and storage.
- Optimize big data infrastructure for performance and scalability.
- Troubleshoot and resolve issues related to big data infrastructure.
- Collaborate with data scientists and other engineers to develop and deploy machine learning models.
Required Skills
- Apache Spark
- Hive
- Apache Iceberg
- Medallion framework
Recommended Jobs
5 jobs available
Project ManagementFull Time8+ years
Agile methodologies
project management
Scrum framework
+1 more
Business AnalysisFull Time10+ years
Mulesoft Anypoint Platform
SAP integration
Agile/Scrum
Risk ManagementFull Time12+ years
Third-Party Risk Management
risk governance
T&O contracts
ITFull Time10-12 years
ServiceNow development
scripting
workflows
+1 more
EngineeringFull-Time5+ years
Core Java
Spring Framework
Spring Boot
+1 more