AWS Data Engineer

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and PySpark.
• Build and manage ETL/ELT workflows for processing structured and unstructured data.
• Develop and optimize Spark applications to handle large-scale datasets efficiently.
• Implement cloud-native data solutions using AWS services.
• Design and maintain Data Lakes on AWS.
• Perform data ingestion from multiple sources including APIs, databases, files, and streaming platforms.
• Monitor, troubleshoot, and optimize data workflows for performance and reliability.
• Ensure data quality, integrity, security, and governance standards are followed.
• Implement automation and orchestration of data workflows using scheduling tools.
• Collaborate with cross-functional teams for requirement gathering and solution design.
• Conduct code reviews, unit testing, and performance tuning.
• Prepare technical documentation and deployment procedures.
• Support production incidents and perform root cause analysis.
  
  

• Strong expertise in Python
• Hands-on experience with PySpark
• Proficiency in SQL and query optimization