Connect to a GPU Container Job

Open an interactive shell into a running GPU Container Job. From the shell, you can run commands directly on the container.

Prerequisites

You need the following before you start:

The CosmicAC CLI installed and configured. See Install the CLI.
A running GPU Container Job. See Create a GPU Container Job or Create a GPU Container Job in the web interface.

Steps

Find the job ID

List your jobs and copy the ID of the job you want to connect to:

cosmicac jobs list

Find the container index

View the job's details, replacing <jobId> with the ID from the previous step:

cosmicac jobs detail <jobId>

The Containers section lists each container with its index, starting at 0. Note the index of the container you want to connect to.

Open the container shell

Open a shell into the container. Replace <jobId> and <containerId> with the values from the previous steps:

cosmicac jobs shell <jobId> <containerId>

Run sudo for root access, and type exit to close the shell.

The shell requires the job's status to be running. If the container is still booting, wait, then try again.

Help and troubleshooting

nvidia-smi fails with Failed to initialize NVML: Unknown Error

If nvidia-smi displays Failed to initialize NVML: Unknown Error, the NVIDIA device files under /dev and /proc/driver/nvidia/version are present, but the GPU is not available to nvidia-smi.

Restart the container from the shell:
```
kill 1
```
Use sudo if required.
Reconnect to the container with cosmicac jobs shell and run nvidia-smi again.

Shell does not open when the job is Running

cosmicac-cli connects to cosmicac-wrk-agent-instance over hyperswarm-ssh. The connection then goes directly to the job's virtual machine.

Some cluster network configurations block this connection. As a result, the shell does not open even though the job reports Running.

To route the connection through a relay, see Set up a relay for CosmicAC.