Databricks QA Tester

Overview

Role Overview

We are seeking a Databricks QA/Tester with 2-4 years of experience to join our data engineering and analytics team. The ideal candidate will have hands-on expertise in Python, PySpark, and Databricks, with a strong focus on testing, validation, and quality assurance of ETL pipelines and distributed data processing workflows.

🔹 Key Responsibilities

• Design and execute test plans, test cases, and automation scripts for ETL workflows built on Databricks.
• Validate data ingestion, transformation, and integration pipelines developed in PySpark and Python.
• Perform functional, regression, and performance testing of large-scale distributed data processing jobs.
• Ensure data quality, consistency, and accuracy across Bronze, Silver, and Gold layers in medallion architecture.
• Collaborate with developers, data engineers, and business stakeholders to define QA strategies.
• Implement automated testing frameworks for Databricks notebooks and workflows.
• Troubleshoot and report defects, ensuring timely resolution and continuous improvement.
• Document QA processes, test results, and best practices.
  
  
  

🔹 Required Skills & Qualifications

• 2-4 years of experience in QA/testing for data engineering or ETL pipelines.
• Strong hands-on skills in Python for scripting and automation.
• Proficiency in PySpark for distributed data validation and testing.
• Experience with the Databricks platform (notebooks, jobs, clusters, Unity Catalog).
• Knowledge of data warehousing concepts, SQL, and relational databases.
• Familiarity with Structured Streaming, Auto Loader, and Delta Lake.
• Strong analytical and problem-solving skills.
• Ability to work collaboratively in agile teams.
  
  
  

🔹 Preferred Qualifications

• Exposure to cloud platforms (Azure, AWS, or GCP).
• Experience with CI/CD pipelines and version control (Git).
• Understanding of data governance, security, and compliance.
• Knowledge of test automation frameworks for big data environments.
  
  
  

Responsibilities

Role Overview

We are seeking a Databricks QA/Tester with 2-4 years of experience to join our data engineering and analytics team. The ideal candidate will have hands-on expertise in Python, PySpark, and Databricks, with a strong focus on testing, validation, and quality assurance of ETL pipelines and distributed data processing workflows.

🔹 Key Responsibilities

• Design and execute test plans, test cases, and automation scripts for ETL workflows built on Databricks.
• Validate data ingestion, transformation, and integration pipelines developed in PySpark and Python.
• Perform functional, regression, and performance testing of large-scale distributed data processing jobs.
• Ensure data quality, consistency, and accuracy across Bronze, Silver, and Gold layers in medallion architecture.
• Collaborate with developers, data engineers, and business stakeholders to define QA strategies.
• Implement automated testing frameworks for Databricks notebooks and workflows.
• Troubleshoot and report defects, ensuring timely resolution and continuous improvement.
• Document QA processes, test results, and best practices.
  
  
  

Requirements

Role Overview

We are seeking a Databricks QA/Tester with 2-4 years of experience to join our data engineering and analytics team. The ideal candidate will have hands-on expertise in Python, PySpark, and Databricks, with a strong focus on testing, validation, and quality assurance of ETL pipelines and distributed data processing workflows.

🔹 Key Responsibilities

• Design and execute test plans, test cases, and automation scripts for ETL workflows built on Databricks.
• Validate data ingestion, transformation, and integration pipelines developed in PySpark and Python.
• Perform functional, regression, and performance testing of large-scale distributed data processing jobs.
• Ensure data quality, consistency, and accuracy across Bronze, Silver, and Gold layers in medallion architecture.
• Collaborate with developers, data engineers, and business stakeholders to define QA strategies.
• Implement automated testing frameworks for Databricks notebooks and workflows.
• Troubleshoot and report defects, ensuring timely resolution and continuous improvement.
• Document QA processes, test results, and best practices.

---