Databricks Engineer - Systems Analyst 3 - 26-09425
Job Description
Job Description
Databricks Engineer – Systems Analyst 3
Location: Austin, Texas
Experience Level: 8+ Years
Job Summary
We are seeking an experienced Databricks Engineer to design, develop, optimize, and maintain scalable data solutions using the Databricks platform.
The ideal candidate will have extensive experience with Databricks, Apache Spark, PySpark or Scala, SQL, Python, Delta Lake, Lakeflow Declarative Pipelines, Medallion Architecture, and data lakehouse solutions.
The candidate will work on complex data engineering initiatives and collaborate with technical and business stakeholders to develop reliable data solutions supporting enterprise analytics, reporting, and decision-making.
Key Responsibilities
- Design, develop, and optimize scalable data solutions using Databricks.
- Build and maintain ETL/ELT pipelines using Apache Spark, PySpark, Python, or Scala.
- Develop and manage Lakeflow Declarative Pipelines (formerly Delta Live Tables/DLT).
- Implement Medallion Architecture using Bronze, Silver, and Gold layers.
- Design and develop enterprise data models for analytics and reporting.
- Implement dimensional data modeling, including Star and Snowflake schemas.
- Work extensively with Delta Lake and modern data lakehouse architectures.
- Build, schedule, monitor, and optimize jobs using Lakeflow Jobs (formerly Databricks Workflows) or tools such as Airflow.
- Implement data quality, validation, governance, and security frameworks.
- Develop Databricks SQL dashboards, Databricks Apps, and analytical solutions.
- Optimize workloads for scalability, performance, reliability, and cost efficiency.
- Analyze business and technical requirements and translate them into effective data solutions.
- Prepare technical documentation covering data requirements, processing logic, and implementation steps.
- Evaluate system capabilities and limitations to determine appropriate technical solutions.
- Present technical findings and analytical insights to technical and business stakeholders.
Required Qualifications
- 8+ years of experience in IT supporting the design, development, deployment, or delivery of technology solutions.
- 8+ years of experience with Databricks, including building and optimizing ETL/ELT pipelines using Apache Spark.
- 8+ years of experience with data warehousing and dimensional data modeling.
- 8+ years of experience with SQL and Python or Scala for large-scale data processing.
- 8+ years of experience developing Databricks-native dashboards and applications, including Databricks SQL dashboards and Databricks Apps.
- 8+ years of experience implementing data governance, data quality, and data security practices.
- 8+ years of experience with Lakeflow Declarative Pipelines / DLT.
- 8+ years of experience with Delta Lake, Medallion Architecture, data lakehouse design, and Lakeflow Jobs / Databricks Workflows or similar orchestration tools.
- Excellent verbal and written communication skills.
- Strong analytical, troubleshooting, and problem-solving abilities.
- Ability to independently handle complex technical assignments.
Preferred Qualifications
- Experience working in public sector or state government environments.
- Databricks certification, such as Databricks Certified Data Engineer Associate or Professional.
- Experience with CI/CD practices for data pipelines.
- Experience with DevOps and Git-based workflows.