TalentAQ

TalentAQ

Big Data Engineer

EngineeringFull TimeRemote

Required Skills
14 skills

Hadoop
Spark
Flink
Kafka
Hive
Presto
Java
Scala
Python
AWS
GCP
Azure
Docker
Kubernetes

Job Description

<p>We are looking for a Big Data Engineer with strong experience in building large-scale distributed data processing systems, data engineering solutions, or high-performance internet-scale applications. The ideal candidate is passionate about solving complex data challenges, optimizing performance, and delivering scalable, reliable systems.</p><p>Key Responsibilities:</p><ul><li>Design, develop, and maintain large-scale distributed data processing pipelines and applications.</li><li>Build and optimize data engineering solutions to support analytics, reporting, and real-time data needs.</li><li>Work with big data technologies, cloud platforms, and distributed systems to ensure high availability and scalability.</li><li>Collaborate with cross-functional teams including Data Scientists, Software Engineers, and Product Managers.</li><li>Ensure data quality, integrity, and security across the ecosystem.</li><li>Troubleshoot and resolve performance bottlenecks in large-scale data applications.</li></ul><p>Required Qualifications:</p><ul><li>Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.</li><li>Strong experience with large-scale distributed data systems (e.g., Hadoop, Spark, Flink, Kafka, Hive, Presto, etc.).</li><li>Expertise in data engineering, ETL pipelines, and data warehousing concepts.</li><li>Solid programming skills in Java, Scala, or Python.</li><li>Experience with cloud platforms (AWS, GCP, Azure) for big data processing.</li><li>Strong understanding of distributed systems, algorithms, and performance optimization.</li><li>Proven track record of building and scaling internet-scale applications or data platforms.</li></ul><p>Preferred Skills (Good to Have):</p><ul><li>Knowledge of real-time streaming systems.</li><li>Familiarity with containerization & orchestration (Docker, Kubernetes).</li><li>Hands-on experience with machine learning pipelines or analytics platforms.</li><li>Strong problem-solving and communication skills.</li></ul>

We are looking for a Big Data Engineer with strong experience in building large-scale distributed data processing systems, data engineering solutions, or high-performance internet-scale applications. The ideal candidate is passionate about solving complex data challenges, optimizing performance, and delivering scalable, reliable systems.

Key Responsibilities:

  • Design, develop, and maintain large-scale distributed data processing pipelines and applications.
  • Build and optimize data engineering solutions to support analytics, reporting, and real-time data needs.
  • Work with big data technologies, cloud platforms, and distributed systems to ensure high availability and scalability.
  • Collaborate with cross-functional teams including Data Scientists, Software Engineers, and Product Managers.
  • Ensure data quality, integrity, and security across the ecosystem.
  • Troubleshoot and resolve performance bottlenecks in large-scale data applications.

Required Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
  • Strong experience with large-scale distributed data systems (e.g., Hadoop, Spark, Flink, Kafka, Hive, Presto, etc.).
  • Expertise in data engineering, ETL pipelines, and data warehousing concepts.
  • Solid programming skills in Java, Scala, or Python.
  • Experience with cloud platforms (AWS, GCP, Azure) for big data processing.
  • Strong understanding of distributed systems, algorithms, and performance optimization.
  • Proven track record of building and scaling internet-scale applications or data platforms.

Preferred Skills (Good to Have):

  • Knowledge of real-time streaming systems.
  • Familiarity with containerization & orchestration (Docker, Kubernetes).
  • Hands-on experience with machine learning pipelines or analytics platforms.
  • Strong problem-solving and communication skills.

Similar Jobs

9914 jobs available

Data & AnalyticsFull Time2-7 years
SQL
Python
Apache Airflow
+14 more
Data Engineering10+ years
Java
PHP
Python
+28 more
Data Structures
Algorithms
Distributed Systems
+20 more
DataWorks
Engineering3+ years
SQL
Python
ETL
+6 more
EngineeringContract
Data Structures
Algorithms
Big Data
+21 more
Data & AnalyticsFull Time9+ years
Data Engineering
Business Systems Analysis
Big Data
+27 more