5.4 Compute Beyond the Laptop

🖥️ Lecture slides — Session 27 (Fri Dec 4)

Everything in this course so far runs on a laptop, on purpose: small data and fast iteration teach more per hour than big data and long queues. But real projects outgrow that. This page is the map for when they do.

When you have outgrown the laptop¶

Two symptoms, one each for memory and time:

Data larger than RAM. Pandas wants your table in memory several times over during a join or a groupby. Once the working set passes a few tens of gigabytes, you either stream (chunked reads, Zarr, Dask/xarray lazy loading) or move to a machine with more memory. Try streaming first; it is free.
Training longer than hours. A model that trains in 10 minutes lets you run the 6-run study of Section 5.2 over lunch. A model that trains in 10 hours makes every mistake cost a day, and hyperparameter search impossible. A single decent GPU often buys a 10-50x speedup over laptop CPU for deep models.

A third, softer symptom: you need to run many independent jobs (per-station processing, seed sweeps, cross-validation folds). That is embarrassingly parallel and is exactly what clusters are for.

Before scaling up, profile. Many “I need a cluster” problems are actually an O(n²) loop, float64 where float32 would do, or reloading data inside the loop. An hour of profiling is cheaper than any GPU.

The options ladder¶

Climb only as far as you need.

Departmental servers and university HPC¶

Usually your first and cheapest stop: UW students have access to Hyak. Clusters run a scheduler, almost always SLURM. You describe the job; the scheduler finds it a node:

sbatch --gres=gpu:1 --mem=64G --time=08:00:00 --wrap "pixi run python train.py --config sweep3.yaml"

Your reproducibility stack transfers as is: clone the repo on the cluster, pixi install (or build an Apptainer container from your image), run the same scripts. If your workflow only works on your laptop, Section 5.1 is not yet done.

NSF ACCESS allocations¶

NSF ACCESS grants free compute time on national systems to US researchers — including GPU nodes far larger than departmental hardware. “Explore” allocations are small, quick to obtain, and sized for a graduate project; your advisor can hold larger ones. If your project is grant-funded research, ask for an allocation before paying a cloud provider.

Commercial cloud (AWS, GCP, Azure)¶

The cloud sells two things you may want: elastic compute (a hundred machines for an hour) and object storage next to public datasets. The pieces to know:

Object storage (S3 / GCS / Azure Blob): flat namespaces of objects addressed by URL, effectively infinite, cheap to store, priced for access and egress. Most modern geoscience archives (Sentinel, ERA5, seismic data on AWS Open Data) live here.
Spot / preemptible GPU instances: the same hardware at a 60-90% discount, with the caveat that the provider can reclaim it with minutes of notice. Fine for training if you checkpoint regularly — which you should anyway.
Cost discipline: estimate before you run (each provider has a pricing calculator: instance price × hours × count, plus storage and egress); set a billing alarm on day one, before the first instance, not after the first surprise; shut things down — an idle GPU instance bills exactly like a busy one; and prefer egress-free access patterns (compute in the same region as the data).

A student who runs an uncapped cloud account learns about cost discipline exactly once. Set the alarm first.

Teaching-scale: Colab and Codespaces¶

Google Colab gives a free (throttled) GPU attached to a notebook, and GitHub Codespaces gives a disposable VS Code machine wired to your repo. Both are fine for coursework, demos, and trying an idea before requesting real resources. Neither is a research platform: sessions are ephemeral, hardware is unpredictable, and long jobs get killed. Treat them as scratch paper.

Restricted environments¶

Everything above assumes open outbound internet: pixi install reaching conda-forge and PyPI, CI on github.com, hosted experiment trackers, anonymous reads from public buckets. National laboratories, secure enclaves, and many government and industry networks block some or all of that. The workflow survives; the transport changes. Four substitutions cover most cases.

Package mirrors and offline installs. The pixi workflow from Chapter 1.3 and the locked environment from 5.1 transfer unchanged — only the channel URLs move. Most restricted sites run an internal mirror of conda-forge and PyPI (Artifactory, Nexus, or a plain file mirror); point pixi at it with a [mirrors] table in its config, and the same pixi.toml + pixi.lock resolve to the same environment. For a fully air-gapped system, resolve and download on a connected machine, move the package cache (or a container image, below) through the approved transfer gateway, and install offline. Ask your facility which path is sanctioned before inventing one.

Container transfer. Build where you have internet, run where you do not. Build the Apptainer image (or docker save a tarball) on an open machine, move the single .sif file in through the gateway, and run it unprivileged on the cluster — no compute node ever needs to reach conda-forge. On restricted HPC this is the primary environment path, not a footnote: one file carries the entire pinned stack from 5.1.

Self-hosted CI. The executable-checks pattern — pinned environment, re-execute everything on every change — is what carries the guarantee in 5.1 and the copyable workflow in 5.3; GitHub Actions is just one host for it. GitLab CI, Jenkins, or a self-hosted runner inside the enclave runs the same job with a different YAML dialect. If github.com is unreachable, an internal GitLab instance with one runner on a lab workstation preserves the property that matters: what counts is what CI can execute.

Hosted vs self-hosted experiment trackers. Weights & Biases as used in 5.2 is a hosted service: every logged parameter, metric, and artifact leaves your network. Before adopting it, run the choice past your data-classification rules — run metadata can be sponsor-restricted or export-controlled even when the code is not. Self-hosted MLflow keeps the same records inside the enclave, and the 30-line JSON tracker built in 5.2 is compliant anywhere, because it only ever writes local files. The same review applies to data access itself: the anonymous public-bucket reads in the next section are outbound calls your network may block, and the fallback is an institutional mirror or a subset pre-staged through the gateway.

Cloud-optimized data access: read in place¶

The old workflow — download the archive, then analyze — breaks when the archive is 100 TB. Cloud-optimized formats fix this by making partial reads cheap over HTTP: Zarr for chunked N-dimensional arrays, Cloud-Optimized GeoTIFF (COG) for rasters, plus Parquet for tables. You open the remote dataset lazily and pull only the chunks your computation touches.

Notebook 5.5 executes this pattern for real, against the 30 TiB MUR sea-surface-temperature Zarr store on AWS open data: lazy open in ~12 s off one 10 KiB metadata read, a ten-day regional subset that moves 60 MiB to produce a 9 MiB answer — one part in ~525,000 of the store — and a streaming monthly mean that never holds more than a few chunks in memory. The pattern to internalize: the query moves to the data, and only results the size of your answer move back. Combined with compute in the same cloud region, this replaces terabyte downloads with megabyte reads. fsspec provides the same trick for almost any storage backend, and pooch (Section 5.3) remains the right tool for the small, versioned files your repo actually depends on.