Job Description
Job DescriptionWe are looking for a Data Engineer to join a contract opportunity with permanent potential, supporting data-intensive work in Madison, Wisconsin. This role focuses on building and optimizing modern data solutions that enable scientific and business teams to access reliable, scalable information. The ideal candidate brings deep experience with cloud-based engineering, strong Databricks expertise, and the ability to work across technical and research-focused stakeholders.
Responsibilities:
• Design, build, and maintain scalable data pipelines that ingest, transform, and deliver complex datasets for analytics and reporting.
• Develop and optimize Databricks solutions using Python, Spark, PySpark, and Delta Lake to support high-performance data processing.
• Create and enhance cloud-based data architecture in Azure, ensuring reliability, maintainability, and efficient data access.
• Partner with cross-functional teams in IT, science, and business to translate research and operational needs into effective data engineering solutions.
• Implement data models and warehousing structures that improve reporting accuracy, usability, and long-term scalability.
• Manage integration of biological, genomic, or other life sciences data sources while preserving data quality and consistency.
• Write and refine database objects such as queries, stored procedures, and functions to support downstream applications and analysis.
• Contribute to Agile delivery practices by participating in planning, prioritization, and iterative solution development.• Bachelor’s degree in Computer Science, Information Systems, Bioinformatics, Computational Biology, or a related discipline; an advanced degree is preferred.
• At least 7 years of experience in data engineering, integration, and reporting, including work with cloud-based data platforms.
• Advanced hands-on experience with Databricks and related technologies, including Python, Spark, PySpark, Spark SQL, and Delta Lake.
• Strong knowledge of relational database development, including query optimization, stored procedures, and user-defined functions.
• Solid understanding of ETL design, data warehousing principles, and best practices for scalable data management.
• Experience with Microsoft Azure; familiarity with Microsoft Fabric is a plus.
• Background working with bioinformatics, genomics, or other omics data in a life sciences or research-driven environment.
• Legal authorization to work in the United States is required.