Data Engineer (Real‑Time Stream) (225-1008)

A Data Engineer specializing in Real-Time Stream processing is crucial for organizations aiming to process and analyze data as it is generated. This role involves designing, implementing, and optimizing data infrastructure and streaming data pipelines to support real-time analytics and decision-making processes. The ideal candidate will have a solid background in software engineering, coupled with experience in distributed systems, data integration, and stream processing frameworks such as Apache Kafka, Apache Flink, or Amazon Kinesis. A proactive approach to problem-solving and a deep understanding of the entire data lifecycle are essential. Joining the team will give you the opportunity to work on cutting-edge projects, driving innovation, and enabling business intelligence efforts at scale. Your contributions will ensure the seamless flow and integrity of information, empowering the organization to respond swiftly to changes and opportunities in the market.

Responsibilities

• Design and implement scalable data pipelines for real-time data processing tasks.
• Collaborate with data scientists and analysts to integrate machine learning models.
• Develop monitoring tools and dashboards to ensure data accuracy and system health.
• Optimize existing infrastructure for enhanced performance and cost efficiency.
• Evaluate and integrate new technologies improving our streaming data capabilities.
• Participate in code reviews to ensure coding and architectural best practices.
• Coordinate with stakeholders to gather requirements and translate them into technical specifications.
• Implement data governance and security policies to protect sensitive information.
• Provide technical guidance and mentorship to junior team members and new hires.
• Document system architectures, processes, and tools for knowledge sharing and training purposes.
• Collaborate with operations teams to troubleshoot and resolve data pipeline issues.
• Manage real-time streaming projects and ensure they are delivered on schedule.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or related technical fields required.
• Proven experience with real-time streaming frameworks like Apache Kafka or Flink.
• Strong proficiency in programming languages such as Java, Scala, or Python.
• Experience with cloud platforms like AWS, GCP, or Azure for deploying data solutions.
• Solid understanding of data modeling, ETL processes, and data warehousing concepts.
• Ability to manage multiple projects and meet tight deadlines effectively.
• Excellent problem-solving skills and a proactive approach to challenges.
• Strong verbal and written communication skills for cross-functional collaboration.