Job Description
Job Description
ComResource is looking for a Data Engineer – Databricks.
Responsibilities:
Data Engineering & Pipelines
- Build scalable, production-grade ETL/ELT pipelines using Databricks, including PySpark, Spark SQL, Delta Live Tables, and Workflows.
- Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.
- Develop optimized transformations, data quality rules, and reusable framework components.
- Leverage built-in AI tooling for faster delivery where applicable.
- Implement best practices for job orchestration, monitoring, alerting, and automation.
Data Modeling
- Design and implement dimensional models including star and snowflake schemas for analytical workloads.
- Apply normalization and denormalization strategies for performance and usability.
- Create physical data models aligned to Delta Lake and Medallion architecture.
- Ensure data quality, integrity, and alignment with enterprise governance policies.
Essentials:
- 7–10+ years of experience in data engineering.
- Proven experience building and deploying large-scale Databricks pipelines.
- Advanced hands-on Databricks experience, including Spark, Delta Lake, Workflows, and Unity Catalog.
- Strong understanding of Medallion architecture (Bronze/Silver/Gold).
- Proficiency with PySpark, SQL, ETL/ELT frameworks, and Delta Lake optimizations.
- Strong SQL programming and performance tuning skills.
- Strong experience working with CI/CD, Git, and job orchestration tools.
- Deep understanding of data modeling principles for warehousing and analytics.
- Experience with cloud environments including AWS, Azure, or GCP.
- Experience with modern data lakehouse concepts and distributed systems.
Req ID: AM71721771