Search

Sr. Principal Engineer, Scale-Up (Data Path Features)

PublishedPublished: 6/14/2022
Technology

Job Description

Job DescriptionWhy join Upscale, Inc.

Upscale, Inc., is building the high-performance infrastructure powering the next generation of artificial intelligence. Backed by over $300M in funding and rapid global adoption, we are scaling systems designed for the world’s most demanding AI workloads.

We focus on first-principles engineering across silicon, systems, and networking—where performance, scale, and execution are critical. Our team is talent-dense and high-performing. We value ownership, technical rigor, and speed, and we offer the opportunity to work on foundational problems with immediate, real-world impact.

If you’re looking to do high-impact work, move fast, and help define the infrastructure behind the future of AI—Upscale is where you can produce meaningful work at the frontier—and operate at a high standard.


Position Overview

Senior individual-contributor validation role. You own how data-path features are qualified on our scale-up AI networking platform, from the SAI/SDK programming path through ASIC forwarding and traffic-management behavior. You set the test strategy, own feature readiness and release sign-off, debug the hardest cross-stack failures, and grow the QA engineers around you. Primary depth is traffic management. Feature scope (cycle-accurate model and silicon)

Key Responsibilities

  • Classification and marking: DSCP and 802.1p to traffic class, trust, TC-to-queue, remarking, port versus switch map binding, classification miss, ACL rewrite

  • Queueing: unicast and multicast queues, counters, occupancy, watermarks, drops

  • Scheduling: strict priority, WRR/DWRR, hierarchy, shaping, rate limiting, fairness under oversubscription

  • Buffering: ingress and egress pools, static and dynamic thresholds, priority groups, headroom, reserved versus shared

  • PFC and lossless Ethernet: pause, headroom, lossless queues, watchdog, storm and deadlock, DCB

  • ECN and WRED: thresholds, drop probability, ECT/CE, interaction with DCQCN under RoCEv2

  • Policing: color-blind and color-aware, CIR/PIR/CBS/PBS, trTCM

  • Forwarding and counters: L2/L3, ECMP, mirror and sample, telemetry that makes the above observable

  • Own the validation strategy, test plans, coverage, and release entry/exit criteria for the features above

  • Qualify lossless and congestion behavior under RoCEv2 and GPU collectives (NCCL/RCCL) at line rate, including incast, PFC headroom, ECN accuracy, and no-drop

  • Build the frameworks and debug tools the team depends on, including model-based checks of classification, queueing, scheduling, and drop or mark outcomes

  • Lead root-cause analysis across software, SDK, model, silicon, and Linux, and decide when the spec, the implementation, or the test is wrong

  • Mentor QA engineers, review their plans and automation, and represent validation in design reviews and the development-to-QA handoff

  • Support customer deployments, proofs of concept, and field issues on data-path and congestion behavior

Requirements

  • 15+ years in networking software, system validation, or ASIC/platform QA, with a substantial stretch as a senior individual contributor

  • Deep hands-on traffic management: PFC, scheduling, buffers, classification and marking, queueing, ECN/WRED

  • Strong L2/L3 and ACL fundamentals, and experience with SAI, a switch SDK, or a NOS such as SONiC or FBOSS

  • Ownership of validation frameworks and automation, not only test execution

  • Python or Go at framework-review level; solid Linux and packet- and counter-level debugging

  • Proven mentoring and influence as an individual contributor, and clear written readiness and root-cause reports

Preferred

  • Ethernet ASIC QA in L2/L3, buffers, scheduling, and queueing

  • RoCEv2, RDMA, DCB, PFC tuning, and DCQCN on AI or HPC fabrics; NCCL/RCCL on NVIDIA or AMD GPU clusters

  • ASIC bring-up, C-model or emulation, and pre-silicon to post-silicon correlation

  • Buffer and headroom sizing; incast, microburst, and oversubscription debug

  • Ixia, Keysight, Spirent, or similar line-rate tools

  • NVLink or ESUN; customer, field, or forward-deployed work is a plus

Where you fall within that range depends on your experience, skills, and impact—we benchmark against internal levels to keep things fair and consistent.

Equal Opportunity

Upscale, Inc., is building a team that reflects a wide range of perspectives, backgrounds, and experiences. We’re proud to be an Equal Opportunity Employer and consider all qualified applicants regardless of race, color, religion, national origin, sex, sexual orientation, gender identity, disability, or veteran status.

Accessibility & Accommodations

We’re committed to making our hiring process accessible to everyone. If you need accommodations at any stage, just reach out to us at hiring@upscaleai.com—we’re happy to help. Note: This inbox is only for accommodation requests.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...