Senior Data Engineer – Databricks

Data Engineer – Databricks

Job Overview

We are looking for an experienced Data Engineer with strong expertise in Databricks to design, develop, and optimize scalable data pipelines and modern data platforms.

The ideal candidate will have hands-on experience with big data technologies, cloud platforms, and ETL/ELT processes, with a strong focus on building reliable, high-quality data solutions that support analytics and business intelligence initiatives.

The Data Engineer will collaborate closely with Data Analysts, Data Scientists, Software Engineers, and business stakeholders to understand requirements and deliver scalable, efficient, and well-governed data solutions.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Build and optimize ETL/ELT workflows for structured and unstructured data.
• Develop and manage data ingestion processes from multiple data sources.
• Process and analyze large-scale datasets using Apache Spark and PySpark.
• Design and maintain data models to support analytics and reporting.
• Optimize data pipeline performance and ensure data accuracy and quality.
• Collaborate with business and technical stakeholders to understand and translate data requirements.
• Implement data governance, security, and engineering best practices.
• Monitor, troubleshoot, and enhance existing data pipelines and data workflows.
• Document technical designs, processes, and data engineering standards.
• Contribute to the continuous improvement of data architecture and engineering practices.

Required Qualifications & Skills

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• 5–7 years of experience in Data Engineering or a similar role.
• Strong hands-on experience with Databricks.
• Strong proficiency in Apache Spark and PySpark.
• Strong SQL skills and experience working with relational databases.
• Hands-on experience developing ETL/ELT data pipelines.
• Experience with at least one major cloud platform: Azure, AWS, or GCP.
• Solid understanding of data warehousing concepts and dimensional modeling.
• Experience with version control systems such as Git.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and collaboration skills.

Preferred Skills

• Experience with Delta Lake and Lakehouse architecture.
• Familiarity with AWS Glue or equivalent cloud data services.
• Experience with orchestration tools such as Apache Airflow.
• Knowledge of CI/CD practices for data engineering.
• Experience implementing data quality, governance, and monitoring solutions.
• Databricks or cloud platform certifications are a plus.

Experience Requirements

• 5–7 years of relevant Data Engineering experience.
• 2–3+ years of hands-on Databricks experience preferred.
• Proven experience building and optimizing production-grade data pipelines.