Azure Databricks Design And Architecture Manager

At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions.

At PwC - AC, as an Azure Architect / Manager, the candidate will interact with Offshore Manager/ Onsite Business Analyst to understand the requirements and the candidate is responsible for end-to-end implementation of Cloud data engineering solutions like Enterprise Data Lake, Data hub in AWS. The candidate should have strong experience in AWS cloud technology. Strong in planning and organization skills. Ability to work as cloud Architect / lead on an agile team and provide automated cloud solutions. Monitor the systems routinely to ensure that all business goals are met as per the Business requirements.

Years of Experience: Candidates with 8-12 years of hands-on experience

Position Requirements

Must Have:

• Proficient with Azure Data Services, Databricks, and Snowflake.
• Experience with deep understanding in both traditional and modern data architecture and processing concepts including relational databases (e.g., SQL Server, MySQL, PostgreSQL), Data warehousing, big data, NoSQL and business analytics
• Rich experience working in Azure ADLS, Data Bricks, Data Flows, HDInsight, Azure Analysis services
• Hands-on experience with Snowflake utilities, SnowSQL, SnowPipe,ETL data Pipelines, Big Data model techniques using
  
  

• Experience in building stream-processing systems, using solutions such as Storm or Spark-Streaming
• Understanding of big data use-cases and Lambda/Kappa design patterns
• Knowledge of real time/stream analytics trends.
• Architecture, design, implementation, and/or support of complex application architectures (i.e. having an architectural sense for connecting data sources, data visualization, structured and unstructured data, etc.)
• Demonstrable hands-on experience implementing Big Data solutions using Microsoft Data Platform and Azure Data Services
  
  

• Exposure to Open-Source technologies such as Apache Spark, Hadoop, NoSQL, Kafka, Solr/Elastic Search
• Drive adoption and rollout of Power BI dashboards, standardizing reporting and enabling self-service BI capabilities for finance stakeholders.
• Should be well versed with quality processes and implementation
• Business / domain knowledge in Financial Services Domain with familiarity with data compliance standards (PCI DSS, GDPR, etc.)
• Experience of working on design and architectural strategy for banking and financial services solutions leveraging Azure and Databricks
• Good knowledge of Application DevOps tools (Git, CI/CD Frameworks) - Experience in Jenkins or Gitlab with rich experience in source code management like Code Pipeline, Code Build and Code Commit
• Guide the assessment and implementation of finance data marketplaces, ensuring seamless integration across business units
• Good communication and presentation skills
• Good understanding of Data Modeling and Data Architecture
  
  

• Experience in building stream-processing systems, using solutions such as Storm or Spark-Streaming
• Experience in Big Data ML toolkits, such as Mahout, SparkML, or H2O
• Knowledge in Python
• Certification on AWS Architecture desirable
• Worked in Offshore / Onsite Engagements
• Experience in AWS services like STEP & Lambda
• Good Project Management skills with consulting experience in Complex Program Delivery
• Good to have knowledge in Cloud - AWS, GCP, Informatica-Cloud, Oracle-Cloud, Cloud DW - Snowflake & DBT
  
  

• BE / B.Tech / MCA / M.Sc / M.E / M.Tech / MBA