Take the product tour
Four short walkthroughs on sample product screens. Use Next and Previous, or the arrow keys. The product interface is English-only.
A node stops responding
-
The cluster summary shows gpu-02 with READY Unknown and the code E-NODE-008.
-
Problems shows E-NODE-008, its age, and runbook RB-NODE-008.
-
The node page opens the coded error card. First field: what happened.
-
No new job lands on the node; running jobs continue.
-
The node is marked Unknown, and the drain lifts by itself when the node returns.
-
The next step for the operator and the offline runbook to follow.
-
The operator can cordon the node to keep it out of the queue during maintenance.
- Cluster summary
- Cluster
- Node profiles
- Nodes
- Onboarding2
- Fabric
- Storage
- Driver stacks
- Tenants
- Queues
- Jobs
- Usage
- Operations
- Approvals1
- Releases & images
- Policies
- Events
- Alarms
- Audit log
- Runbooks
- E-NODE-008gpu-02Node unreachable10 mindrained, takes no new work
- E-PROV-007gpu-03Configuration drift that requires downtime6 minmaintenance window
- train-h20092%6 pending
- train-b30071%2 pending
- inference-l40s48%0 pending
- E-NODE-008gpu-02Node unreachable
- E-PROV-007gpu-03Configuration drift that requires downtime
RB-NODE-008
- Cluster summary
- Cluster
- Node profiles
- Nodes
- Onboarding2
- Fabric
- Storage
- Driver stacks
- Tenants
- Queues
- Jobs
- Usage
- Operations
- Approvals1
- Releases & images
- Policies
- Events
- Alarms
- Audit log
- Runbooks
- What happened
- The node's lease has not been renewed for 60 s.
- Impact
- The node is drained in BareStax Core: no new job lands on it, running jobs continue.
- What BareStax did
- Ready=Unknown was set; the drain is lifted when the node returns.
- What to do
- Check that the node is powered on and barestaxd is running:
systemctl status barestaxd - Runbook
- RB-NODE-008
- Lease lost 10 min ago
- BMC reachable
- XID none
- ECC ok
- NVLink ok
- InfiniBand unknown · data stale
HGX H200 · 8 GPU slots
An alert, its impact, and a two-step approval
-
E-PROV-007 on gpu-03: a change that BareStax will not apply while jobs run.
-
The impact view separates what is confirmed from what is only possible, with the age of the data.
-
Jobs 4521 and 4533 run on gpu-03.
-
The approver must read the impact before continuing.
-
The approver types gpu-03 to confirm, so a wrong target is hard to approve. Try it.
- Cluster summary
- Cluster
- Node profiles
- Nodes
- Onboarding2
- Fabric
- Storage
- Driver stacks
- Tenants
- Queues
- Jobs
- Usage
- Operations
- Approvals1
- Releases & images
- Policies
- Events
- Alarms
- Audit log
- Runbooks
- E-NODE-008gpu-02Node unreachable10 mindrained, takes no new work
- E-PROV-007gpu-03Configuration drift that requires downtime6 minmaintenance window
- train-h20092%6 pending
- train-b30071%2 pending
- inference-l40s48%0 pending
- E-NODE-008gpu-02Node unreachable
- E-PROV-007gpu-03Configuration drift that requires downtime
RB-NODE-008
| JOBID | TENANT | USER | STATE | GPU | IMPACT |
|---|---|---|---|---|---|
| 4521 | research-a | researcher-1 | RUNNING | 4 | Verified |
| 4533 | research-a | researcher-2 | RUNNING | 4 | Verified |
- E-PROV-007 Configuration drift that requires downtime Show impact
A researcher starts a Jupyter session
-
The portal form asks for a queue, the number of GPUs, and the duration.
-
The session is submitted as a scheduled job.
-
The job waits with a visible reason, then runs on gpu-01 and opens in the browser.
-
Every portal action can be copied as a command for scripts.
- My jobs
- Submit job
- Sessions
- Files
- Terminal
- Quota
- Images
| JOBID | NAME | QUEUE | STATE | GPU | NODE | END |
|---|---|---|---|---|---|---|
| 4820 | jupyter | train-h200 | RUNNING | 1 | gpu-01 | 14:06 |
A container image needs two people
-
operator-1 asks to promote a scanned image.
-
The requester cannot approve their own request.
-
A second person, approver-2, reviews and approves.
-
A third person approves; the image is signed with the cluster key and becomes available to compute nodes.