Job Summary:
We are looking for a highly skilled *Confluent Kafka Developer / Streaming Engineer* with 5-8 years of experience in building and maintaining real-time streaming platforms using *Confluent Kafka. This role involves working with **Kafka Streams (KStream), **KSQL/ksqlDB, **Kafka Connect*, and managing Confluent-based infrastructure to enable scalable, secure, and resilient event-driven systems.
Key Responsibilities:
- Kafka Stream Processing
- Design and develop real-time applications using *Kafka Streams (KStream)* for stateless and stateful transformations, windowing, and joins.
- Create *KSQL/ksqlDB* streams and tables for on-the-fly analytics and streaming ETL use cases.
- Optimize streaming pipelines for *throughput, **latency, and **exactly-once processing guarantees*.
- Confluent Platform Integration
- Implement *Kafka Connect* for source/sink connectors (e.g., JDBC, S3, Elasticsearch).
- Manage *Schema Registry* for Avro/JSON/Protobuf schema evolution with full compatibility controls.
- Utilize *Confluent Control Center* for visibility into throughput, lag, and health of data pipelines.
- Provision and maintain *Confluent Kafka clusters* on *Kubernetes, **OpenShift, or **AWS/GCP* using *Helm, **Terraform, or **Ansible*.
- Configure *multi-region replication, **disaster recovery, and **mirror-maker 2.0*.
- Monitor and troubleshoot clusters using *Prometheus, **Grafana, **Confluent Metrics Reporter, or third-party tools (e.g., **Dynatrace, **Alma*).
Required Skills & Experience:
- 5-8 years of experience with *Apache Kafka, including 2-4 years on **Confluent Kafka Platform*.
- Strong hands-on experience with *KStream, **KSQL, and **Kafka Connect*.
- Familiarity with *distributed systems, **schema evolution, **data consistency, and **idempotency*.
- Cloud-native Kafka deployments experience (AWS MSK, Confluent Cloud, or Kubernetes).
- Strong knowledge of *Kafka security, **topic design, and **capacity planning*.
Preferred Qualifications:
- Confluent Kafka certification (Developer/Admin) is a plus.
- Experience in *event-driven microservices, **IoT, or **real-time analytics*.
- Familiarity with *Confluent REST Proxy, **Kafka Streams testing libraries, and **ksqlDB UDF/UDAF*.