Tier 1 NOC Engineer (HPC & Bare-Metal Infrastructure)
Job Description
We are seeking a technical, detail-oriented Tier 1 NOC Engineer to join our 24/7 Operations team. In this role, you will serve as the first line of defense for our global high-performance computing (HPC) bare-metal infrastructure, high-speed InfiniBand fabrics, and parallel storage systems. You will actively monitor cluster telemetry, perform rapid triage on live alerts, execute initial Linux OS and hardware troubleshooting, and coordinate on-site remote-hands data center technicians to maintain sub-15 minute SLA response targets.
\n
Core Responsibilities
\n
1. 24/7 Monitoring & Alert Triage
\n
- \n
- Actively monitor global HPC clusters, bare-metal GPU/CPU nodes, network fabrics, and storage arrays using Grafana, Prometheus, and internal telemetry tools.
- Validate, categorize, and perform initial incident triage for automated alerts and customer ticket submissions.
- Ensure strict adherence to response time SLAs for critical hardware, OS, and network outages.
\n
\n
\n
\n
2. Hardware & Infrastructure Troubleshooting
\n
- \n
- Perform initial diagnostics on x86 bare-metal servers, inspect Out-of-Band (OOB) management interfaces (IPMI, Redfish, iDRAC/iLO), and review hardware logs.
- Identify component-level hardware failures, including GPU/PCIe errors, DIMM faults, drive dropouts, power supply failures, and thermal anomalies.
- Open, track, and direct vendor dispatch tickets (e.g., Supermicro, Dell, HPE, NVIDIA) for physical hardware replacements.
\n
\n
\n
\n
3. Linux OS & Network Isolation
\n
- \n
- Access non-responsive or degraded hosts via serial-over-LAN/SSH to inspect system logs (dmesg, syslog, journalctl).
- Check basic Layer 1–Layer 3 network connectivity, interface link states, IP assignment, and optical transceiver signal levels.
- Escalate complex fabric routing, InfiniBand subnet manager, or parallel storage issues to Tier 2/3 HPC and Network Engineers with clean, structured diagnostic notes.
\n
\n
\n
\n
4. Remote-Hands Coordination & Dispatch
\n
- \n
- Direct on-site data center technicians for physical cable pulls, server reboots, drive swaps, and component replacements.
- Track hardware RMA lifecycles from fault identification to physical repair verification and host re-commissioning.
\n
\n
\n
5. Incident Documentation & Runbooks
\n
- \n
- Maintain detailed, real-time ticket notes documenting all troubleshooting steps, root causes, and resolutions.
- Follow established operational runbooks and assist in updating Standard Operating Procedures (SOPs) as new issues emerge.
\n
\n
\n
Key Qualifications
\n
- \n
- Experience: 1–3 years in a Network Operations Center (NOC), data center technical support, or system administration role.
- Linux Proficiency: Solid hands-on experience navigating and troubleshooting Linux operating systems (Ubuntu, RHEL, Rocky Linux) via command line.
- Hardware Fundamentals: Clear understanding of enterprise x86 server architecture, IPMI/BMC management, RAID arrays, PCIe topologies, and component-level diagnostics.
- Networking Basics: Solid grasp of TCP/IP networking, DNS, VLANs, subnetting, and basic physical layer troubleshooting (copper, fiber optics, transceivers).
- Communication & Shift Work: Excellent written and verbal communication skills; comfortable working rotational, night, or weekend shifts in a 24/7 environment.
\n
\n
\n
\n
\n
\n
Nice-to-Have / Preferred Skills
\n
- \n
- Exposure to High-Performance Computing (HPC) environments or large-scale GPU clusters.
- Basic familiarity with InfiniBand fabrics, RDMA/RoCE, or parallel storage systems (Weka, Lustre, GPFS).
- Experience with monitoring and ticketing systems like Grafana, Prometheus, Datadog, Jira, or ServiceNow.
- Python or Bash scripting capabilities for task automation.
\n
\n
\n
\n
\n