Job Summary

We are seeking a Sr. Developer with 8 to 12 years of experience to join our team. The ideal candidate will have strong technical skills in Spark in Scala Delta Live Pipelines Azure Data Lake Store Azure DevOps Python Databricks SQL Databricks Workflows and PySpark. Additionally experience in Asset & Wealth Management is mandatory. This is a hybrid work model with day shifts and no travel required.

Responsibilities

• Develop and maintain scalable data pipelines using Spark in Scala and Delta Live Pipelines

• Implement and manage data storage solutions using Azure Data Lake Store

• Utilize Azure DevOps for continuous integration and continuous deployment (CI/CD) processes

• Write efficient and maintainable code in Python for data processing tasks

• Design and optimize SQL queries in Databricks SQL for data analysis

• Create and manage workflows in Databricks Workflows to automate data processing tasks

• Develop and maintain PySpark applications for large-scale data processing

• Collaborate with cross-functional teams to gather and analyze requirements

• Ensure data quality and integrity throughout the data lifecycle

• Monitor and troubleshoot data pipeline performance and reliability

• Provide technical guidance and mentorship to junior developers

• Stay updated with the latest industry trends and best practices in data engineering

• Contribute to the continuous improvement of development processes and methodologies

Qualifications

• Possess strong experience in Spark in Scala and Delta Live Pipelines

• Demonstrate expertise in Azure Data Lake Store and Azure DevOps

• Have proficiency in Python and PySpark for data processing tasks

• Show advanced knowledge of Databricks SQL and Databricks Workflows

• Exhibit experience in Asset & Wealth Management domain

• Display excellent problem-solving and analytical skills