· 4 min read · Gaia Lab
Serving models and looking after the cluster: the infrastructure behind the other lines
Closing the series with what is not a research line but makes them possible: open models served as Slurm jobs for others to use, hardware validation, the tests prior to enabling containers and the personal experiments that peek in around the edges.
Ninth and final instalment of Cluster X-ray. Here we group what does not fit into a research line but shows up in almost everyone’s scripts.
Serving models so that others can work #
Several lines in the series need a large model up and running: a judge to score responses, a synthetic data generator, a coding assistant. On a cluster with one GPU per node, the natural way to serve it is a Slurm job that brings up vLLM, waits for it to respond, registers with a shared proxy and stays there until its time runs out. On finishing, it deregisters: the model appears in and disappears from the catalogue along with the job.
What was served says a lot about what is being researched: gpt-oss 120B and Qwen3.6 27B as general models and Qwen3 Coder 30B for code. All open, almost all with their size chosen for the available card: “the model takes up ~66 GiB of the 94 GB; the conservative settings left a lot of headroom, so we opened up the throttle”.
Validating the hardware #
After an expansion in June, all the GPUs went twice through a simultaneous stress test: ten minutes of 32,768×32,768 matrix multiplications with 75% of the memory occupied, logging utilisation, power and temperature every second, with a variant that also loads all the CPU cores. It is the way to know that a new node can take what the other lines’ jobs are going to ask of it.
Before enabling containers #
The 52 jobs from a single day in September are probes lasting seconds, aimed node by node: that the environment responds on the fourteen GPU nodes and the CPU one, that identity resolution works, that a network diagnostics image starts from within a job. They coincide with a Docker pilot on the ARM nodes and with attempts to launch a hardware synthesis tool. It is the methodical verification that precedes offering containers to users.
Around the edges #
The archive also records what people do with the cluster outside their line: an analysis of 6,613 audio tracks with CLAP, Demucs and MERT to extract tempo, key, mood and embeddings; photogrammetry with Meshroom and image-to-3D with Hunyuan3D for a smart mirror; a confidential virtual machine with SEV-SNP whose lifetime is that of the job; and, at the end of September, the first steps of a single-cell genomics line with Geneformer.
On the cluster #
Closing the series #
Nine posts later, the picture of the cluster is the one the overview post anticipated, with the nuances that only the scripts provide: the line that reserves the most GPU-hours does not train, it measures; the models other lines need are served as just another job; and the research questions range from the safety of a language model to the chlorophyll of a lagoon, by way of proteins, forests and teeth.
Keep the scripts.
Figures from Slurm accounting (reserved capacity, not measured usage) and the archived sbatch files. Anonymised post: no identifiable users, paths, emails or project names. Quotations are from the script comments.