Data Engineer

Company: KGIS

Website: Visit Website

Business Type: Enterprise

Company Type: Service

Business Model: Others

Funding Stage: Pre-seed

Industry: IT Services

Salary Range: ₹ 20-30 Lacs PA

Job Description

About Us

At KGiNVICTA, we build cutting-edge digital and analytics solutions for global enterprises, combining domain expertise with advanced technology. Join our team to lead and deliver data-driven innovations that transform business outcomes.

Role Overview

We are looking for a seasoned Big Data Engineer with 5–8 years of hands-on experience in designing, developing, and optimizing large-scale data processing systems. You’ll play a key role in building scalable data pipelines, driving performance tuning, and leading cloud-native big data initiatives.

Key Responsibilities

• Design and develop robust, scalable Big Data solutions using Apache Spark and related technologies.
• Build batch and real-time data pipelines using Spark, Spark Streaming, Kafka, and RabbitMQ.
• Implement ETL processes to ingest and transform data from RDBMS, ERP systems, and file-based sources.
• Optimize Spark jobs for performance and scalability.
• Work with NoSQL technologies like HBase, Cassandra, or MongoDB.
• Query large datasets using tools like Hive and Impala.
• Ensure seamless integration of data from heterogeneous sources.
• Lead a team of data engineers and ensure adherence to Agile methodologies.
• Collaborate with cross-functional teams to deliver cloud-based solutions on AWS or Azure (Databricks preferred).
  
  

• Deep expertise in Apache Spark and distributed computing.
• Strong programming skills in Python.
• Solid experience with Hadoop v2, MapReduce, HDFS, Sqoop.
• Real-time stream processing using Apache Storm or Spark Streaming.
• Proficient in working with messaging systems like Kafka or RabbitMQ.
• SQL mastery: joins, stored procedures, query optimization, and schema design.
• Hands-on with NoSQL databases – HBase, Cassandra, MongoDB.
• Experience with cloud-native services in AWS or Azure.
• Strong understanding of ETL tools and performance tuning.
• Agile mindset and experience working in Agile/Scrum teams.
• Excellent problem-solving skills and team leadership experience.
  
  

• Exposure to data lake and lakehouse architectures.
• Familiarity with DevOps tools for CI/CD and data pipeline monitoring.
• Certifications in cloud or big data technologies (AWS, Azure, Databricks, etc.).
  
  

• Work on innovative projects with Fortune 500 clients.
• Fast-paced, meritocratic culture with real ownership.
• Access to cutting-edge tools and technologies.
• Collaborative, growth-focused environment.