Unify Technologies - Spark Scala Developer - Data Engineering

Spark Core/Streaming with programming language Scala also in Machine Learing OR AI, Good Understanding of Data Structures, Algorithms, Data Transformation, Data Ingestion, Optimization mechanism/techniques, Good understanding of Big Data (Hadoop, MapReduce, Kafka, Cassandra) Technologies

Job Summary : We are looking for a candidate with strong experience into Spark Core/Streaming, Spark RDD API, Spark Dataframes API with Any programming language Scala/ Java/ Python, with Machine Learning and AI, Should have hands-on programming experience and Good Understanding of Data Structures, Algorithms, Data Transformation, Data Ingestion, Optimization mechanism/techniques, Good understanding of Big Data (Hadoop, MapReduce, Kafka, Cassandra) Technologies.

Key Responsibilities

• Design and develop scalable and high-performance data processing applications using Apache Spark (Core, Streaming) with Scala.
• Work with Spark RDD and DataFrame APIs to build robust data pipelines.
• Apply machine learning or AI techniques to solve real-world data problems.
• Perform data ingestion from various structured and unstructured sources.
• Optimize Spark jobs for performance and resource usage.
• Collaborate with cross-functional teams including data scientists, data engineers, and architects.
• Ensure high code quality through unit testing and code reviews.
• Design and implement data transformation, data ingestion, and ETL pipelines.
  
  

• 4+ years of experience in Big Data technologies with strong hands-on development in Apache Spark (Core/Streaming).
• Strong proficiency in Scala (preferred), with knowledge of Java or Python.
• Solid understanding and experience with Spark RDD API and DataFrame API.
• Working experience with Machine Learning or AI frameworks and use cases.
• Strong understanding of data structures, algorithms, and performance optimization techniques.
• Experience with data ingestion, transformation, and processing pipelines.