Job Description:
Infosys is looking for a highly skilled Senior Databricks Consultant with 6 8 years of experience in building scalable data engineering solutions
The ideal candidate should possess strong expertise in Databricks PySpark Apache Spark Python SQL ETL development and modern Data Engineering practices
Key Responsibilities:
Design develop and maintain scalable Databricks based data engineering solutions
Build and optimize complex ETL ELT pipelines using Databricks PySpark Spark SQL and Python
Develop robust data ingestion and transformation frameworks for structured and unstructured data
Implement and manage Delta Lake and Lakehouse Architecture solutions
Design monitor and optimize enterprise grade data pipelines for performance scalability and reliability
Perform performance tuning and troubleshooting of Databricks and Apache Spark workloads
Develop reusable frameworks and best practices for Data Engineering and Data Integration
Collaborate with architects analysts and business stakeholders to translate business requirements into technical solutions
Ensure data quality governance security and compliance standards are followed
Technical Requirements:
6 8 years of experience in Data Engineering and Big Data technologies
Strong hands on experience with Databricks and PySpark
Expertise in Apache Spark Spark SQL and distributed data processing
Strong programming skills in Python and SQL
Experience building and managing enterprise scale ETL ELT pipelines
Hands on experience with Delta Lake and Lakehouse Architecture
Strong understanding of Data Warehousing Data Modeling and Data Integration concepts
Experience working with large scale structured and unstructured datasets
Preferred Skills:
Technology->Data Engineering->Databricks,Technology->OpenSystem->Python - OpenSystem->Python