Deploying an Instant Cluster

Deploying an instant cluster

In your DataCrunch cloud dashboard, select Clusters->Deploy cluster. In the next screen, you can choose your contract duration (starting from 1 day) and the number of GPUs (depending on the available resources) you would like to have in your cluster.

Select the operating system and CUDA version you want to use (we recommend CUDA 12.9):

Next, select your shared filesystem size. File systems are mounted as follows:

• Local storage is mounted to /mnt/local_disk on each worker node.

• SFS is mounted to /home on all nodes, including the jump host.

You also need to supply your SSH public key before you deploy. We recommend you choose the cluster hostname appropriately, since your worker nodes will inherit the hostname as the prefix.

Once the above steps are done, you deploy the cluster, just like you would an ordinary DataCrunch cloud instance.

Accessing your cluster

Once deployment has been done on DataCrunch cloud dashboard, please give the cluster around 20 minutes to start.

Please also note that the jump host node will become accessible a few minutes before the worker nodes are ready, when starting the cluster for the first time.

The default Linux user for your on-demand cluster is ubuntu

Once the cluster has been created, you can proceed to log in by copying the ssh ubuntu@CLUSTER_IP command from the Clusters screen in your dashboard.

You can login to the individual worker nodes from your jumphost by running ssh WORKER_NAME

You can use tab-completion with SSH to quickly login to your worker nodes

Running jobs

We recommend using Slurm to run jobs on the cluster.