Data Engineer

Project Role : Data Engineer

Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.

Must have skills : Apache Kafka

Good to have skills : NA

Minimum 2 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Job Title

Kafka Developer (Apache Kafka / Streaming Engineer)

Key Responsibilities

Design, develop, and maintain scalable Kafka-based data pipelines and real-time streaming applications

Build and manage Kafka producers, consumers, topics, partitions, and consumer groups

Monitor, tune, and optimize Kafka cluster performance, throughput, and latency

Troubleshoot and resolve issues related to data processing, message delivery, and system integrations

Collaborate with application teams, data engineers, and DevOps teams to deliver robust integration solutions

Develop and maintain Kafka Streams, KSQL, and Kafka Connect (source/sink connectors) implementations

Ensure high availability, reliability, fault tolerance, and data consistency across Kafka platforms

Implement security configurations, schema management (Avro/Protobuf), and data governance standards

Create and maintain technical documentation for Kafka configurations, workflows, and architecture

Provide production support, defect analysis (RCA), and performance tuning for Kafka-related applications

Perform root cause analysis (RCA) for Kafka-related defects and integration failures

Manage topic creation, schema evolution, and versioning strategies

Work on event-driven architecture and microservices integration

Qualifications

Bachelor s degree in computer science / engineering / IT or related field

3–5 years of experience in Kafka or real-time data streaming roles

Preferred Experience

Experience in event-driven architecture and microservices design

Exposure to CI/CD pipelines and DevOps tools (Docker, Kubernetes)

Experience with monitoring tools like Prometheus, Grafana

Experience in high-volume transaction systems or telecom/finance domains

Key Deliverables

Stable and scalable Kafka pipelines

Low-latency data streaming solutions

Effective troubleshooting and defect resolution

Documentation and knowledge sharing

Work Environment

Agile development setup with cross-functional teams

High dependency on integration systems (APIGEE, BWCE, microservices, etc.)