Data Engineer, Senior Consultant
Job Description
Job Description
This is Us:JS Perkins Consulting (JSPC) delivers value added management and technology consulting services. JSPC is committed to creating trusted partnerships to provide sustainable solutions to meet our client needs. We thrive on respect for the individual and host an open idea meritocracy.Here is the Role: J.S. Perkins Consulting is seeking a Data Engineer to support a mission-critical Defense Health data modernization initiative.The program is building an enterprise data orchestration layer across a large, federated defense health data landscape. The work connects clinical, operational, financial, and readiness source systems to a modern enterprise data platform, then curates that raw data into governed, well-documented silver and gold data products that mission users can trust and reuse. The model is federated: data stays with its authoritative owners, and the orchestration layer makes it discoverable, interoperable, and consumable through catalogs, APIs, and a data product marketplace.This is a hands-on build role. You will work directly in Spark and Databricks to land data from clinical and operational source systems, transform it into curated silver and gold datasets, and make those datasets reliable enough that mission users trust them without checking your work. The right candidate is strong in SQL, comfortable in distributed processing, disciplined about data quality, and willing to sit down with a subject matter expert to figure out what a messy source field actually means.What do we offer: Medical/Dental/Vision 401K (up to 5% match)11 Paid Holidays4-5 weeks PTO (depending on level)Bonus IncentivesProfessional DevelopmentTeleworkContinuing Education CreditYour Responsibilities: Design, build, and maintain production data pipelines using Apache Spark with Python and SQL in a Databricks environmentIngest and integrate data from diverse defense and federal health source systems across clinical, operational, financial, and readiness domainsTransform raw source data into curated silver and gold data products with documented business logic, clear ownership, and consistent definitionsBuild and maintain API-based integrations for ingesting and exchanging data across federated systems and platformsImplement data quality controls, including rule creation, anomaly detection, and validation checks that hold data integrity and consistency across every layer of the architectureOptimize Delta Lake tables and Databricks workloads for performance, cost, and reliabilityMonitor, troubleshoot, and tune scheduled workflows and production pipelines, resolving failures and performance issues at the root causeProduce and maintain the metadata, lineage, and documentation needed to register data products in the enterprise data catalog and marketplaceEngage subject matter experts directly to decode complex business logic and translate domain knowledge into technical data requirementsContribute to code reviews, automated testing, version control practices, and reusable engineering patterns that raise consistency across the teamManage multiple concurrent data workstreams in a fast-paced delivery environment and communicate status, risks, and blockers earlyWhat you bring: 3 to 10 years of data engineering, software engineering, or closely related technical experience, with 5 or more years preferredExpert-level SQL. Able to write, optimize, and debug complex queries for large-scale analysis and transformation, including complex joins, window functions, and common table expressionsStrong hands-on experience with Apache Spark, processing large datasets efficiently across distributed clustersHands-on experience with the Databricks platform, including workspace management, notebook collaboration, and Delta Lake optimizationStrong proficiency with PythonDemonstrated experience building, monitoring, and maintaining robust end-to-end ETL and ELT pipelines in productionSolid understanding of data quality and data governance practice, including creating data quality rules and implementing anomaly detection to protect data integrity and consistencyWorking knowledge of data modeling and turning raw data into reusable analytics datasets and data productsExperience integrating or consuming data through REST APIsExperience with Git-based development workflows and modern software engineering practicesComfort working in terminal and command-line environmentsAbility to proactively engage subject matter experts, decode complex business logic, and turn domain knowledge into technical data requirementsStrong communication skills, including the ability to explain technical data concepts to non-technical stakeholders and deliver solutions that meet a stated business needDemonstrated ability to turn messy data into structured, high-value assets that support informed decision-makingAbility to manage multiple data workstreams at once, navigate ambiguity, and drive assigned work to resolution without close supervisionBachelor's degree in Computer Science, Engineering, Data Science, or a related technical fieldU.S. citizenship, and the ability to obtain and maintain a DoW Secret clearanceTravel Requirements: Must be willing to travel for client business as needed. Physical Requirements: Work is performed in hybrid/remote office environment and requires operating standard office equipment and keyboards. Must be able to lift small packages or other items up to 20 lbs and travel short or long distances in a car, airplane, or other vehicle of transportation to client destinations nationally and internationally if needed. Preferred Experience:Active DoW Secret clearance or higher at time of applicationGitLab experience, including code versioning, peer review, and managing CI/CD deployment pipelinesAWS or other cloud platform experienceExperience with Advana, the War Data Platform, Palantir Foundry, or similar DoW data environmentsExperience with very large datasets, including datasets containing billions of recordsExperience with defense health, federal health, financial, or other regulated and sensitive data, including PHI and PII handling under HIPAAFamiliarity with health data standards such as HL7 or FHIR, ICD-10, or CPTExperience with data catalog, metadata, and lineage toolingExperience using AI-assisted development tools to accelerate routine engineering, testing, documentation, and debuggingCulture Statement:At JS Perkins Consulting, we believe our people are the foundation of our success. Because of this, we value talent and drive, celebrate individuality, embrace diversity, and inspire authenticity. We have a culture of mutual trust and respect and believe everyone has a voice. JSPC goes above and beyond for the client and each other.EEO Statement:JSPC is an equal opportunity employer that is committed to diversity and inclusion in the workplace. We prohibit discrimination and harassment of any kind based on race, color, sex, religion, sexual orientation, national origin, disability, genetic information, pregnancy, or any other protected characteristic as outlined by federal, state, or local laws. This policy applies to all employment practices within our organization. JS Perkins Consulting makes hiring decisions based solely on qualifications, merit, and business needs at the time.