Technical Data Engineer (Databricks)
ShyftLabsCoimbatoreFull TimeEngineering
Remotely
sqlmedallion architecturedatabrickspysparkrest apidelta lakeetl/eltunity catalog
Job Description
📋 Description
- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and SQL.
- Integrate data from databases, Amazon S3, files, and REST APIs.
- Build data pipelines with Databricks Unity Catalog and Delta Lake.
- Manage Delta Lake tables with Medallion Architecture (Bronze, Silver, Gold).
- Create, schedule, monitor, and optimize Databricks Jobs and Workflows.
- Collaborate with cross-functional teams to deliver production-ready data solutions.
🎯 Requirements
- Strong expertise in Python, PySpark, and Advanced SQL.
- Hands-on with Databricks Lakehouse Platform; Unity Catalog, Delta Lake, Workflows/Jobs, Clusters
- Experience with REST APIs for data ingestion/export; ETL/ELT development; batch processing; data
- Knowledge of data warehousing concepts, structured/semi-structured data (CSV, JSON, Parquet
- Git and CI/CD best practices.
🎁 Benefits
- Competitive salary with strong insurance package.
- Growth opportunities and extensive learning resources.
Back to all jobs