Data Engineer

Job Description

Job Title: Data Engineer

Job Location: Bangalore

Job Role

Works with business stakeholders to model their data landscape, obtains data extracts and defines secure data exchange approaches. Explores suitable options, designs, and creates high performance data pipelines (data lake / data warehouses) for specific analytical solutions

Roles & Responsibilities

• Responsible for the design, development, and implementation of the ETL jobs for ingestion and transformation areas of the data pipeline
• Works closely with Data Scientists to ensure data quality and availability for analytical modelling
• Collaborates with Data Scientists to map data fields to hypotheses and curate, wrangle, and prepare data for use in their advanced analytical models.
• Acquires, ingests, and processes data from multiple sources and systems into Big Data platforms
• Understanding, assessing and mapping the data landscape.
• Maintaining our Information Security standards
• Defining the technology stack to be provisioned by our infrastructure team.
• Building modular pipeline to construct features and modelling tables.
  
  

• Bachelor’s degree in computer science, BCA / MCA; Masters degree preferred
• Experience in data pipeline software engineering and implementing best practice in python - linting, unit tests, integration tests, git flow/pull request process, objectoriented development, data validation, algorithms and data structures, technical
• troubleshooting and debugging, bash scripting.
• Experience preparing data for analytics and following a data science workflow
• Experience with analytics (descriptive, predictive, EDA), feature engineering and python visualization libraries – e.g., matplotlib, seaborn or other
• Comfortable with notebook and source code development – Jupyter, Pycharm/VScode
• Ability to work under pressure with a solid sense for setting priorities
• Excellent interpersonal, leadership and communication skills
• Strong command of English language (both verbal and written)
  
  

• Big Data / Database platforms like Hadoop, hbase, CouchDB, hive, Pig, TeraData, Oracle, MySQL etc
• Visualization tools like Tableu, QlikView or similar reporting and BI packages
• CDC tools (Qlik Replicate preferred)
• Used SQL, PL/SQL and similar languages, UNIX shell scripting