Databricks Engineer - Senior/ Lead

📋 Description

• Design, develop, and maintain scalable data pipelines and data engineering solutions using Azure
• Build robust ETL/ELT workflows using Python, PySpark, Spark SQL, and Apache Spark.
• Develop SQL queries and transformations for analytics and ML use cases.
• Optimize Spark jobs for performance, scalability, reliability, and cost efficiency.
• Work with Parquet, Delta, JSON, and CSV data formats.
• Design and maintain Delta Lake tables with ACID, time travel, and schema evolution.

🎯 Requirements

• 6+ years of professional Data Engineering experience.
• Hands-on Azure Databricks design and development expertise.
• Advanced Python for data processing and pipelines.
• Strong SQL with complex joins, window functions, and performance tuning.
• Apache Spark and PySpark distributed processing experience.
• Delta Lake knowledge (ACID, time travel, schema evolution).

🎁 Benefits

• Senior/Lead-level scope with influence on engineering standards and architecture.
• Work within AI-focused, cloud data platforms environment.
• Hands-on with Azure Databricks, Spark, Delta Lake, and modern cloud data tech.
• Opportunity to contribute to analytics and ML use cases with data scientists and analysts.
• Exposure to CI/CD, cloud security, data governance, and streaming data tech.
• Collaborative environment across data engineering, analytics, data science, and stakeholders.