This document provides a comprehensive overview of the GPU workstation’s administration, configuration, and user guidelines. Administrators: * Pascal Tribel: pascal.tribel@ulb.be * Gian Marco Paldino: gian.marco.paldino@ulb.be * Cédric Simar: cedric.simar@ulb.be
-
1. Onboarding and First-Time Access
Before you can connect to the server, you must have an account created for you. Please follow these steps:
- Contact the Administrators: Send an email to one of the administrators listed below to request a username.
- Provide Your Public SSH Key: In the same email, you must include your public SSH key (id_rsa.pub or similar). Password-based login is disabled for security, so key-based authentication is required. For instructions on how to create a key-pair, you can look here.
Once your account has been created, you can proceed to the next step.
-
2. Announcement and Issue Reporting
All announcements regarding the GPU workstation (e.g., maintenance, reboots) are posted on the GPU Workstation Teams channel. This channel is also the designated place for reporting any issues you encounter.
Link: https://teams.microsoft.com/l/channel/19%3a91677eba27174f00bc02a1e6dd82675d%40thread.tacv2/Gpu%2520Workstation?groupId=99445ba5-ff9f-4bab-a0f3-ecd5d73dd8fe&tenantId=30a5145e-75bd-4212-bb02-8ff9c0ea4ae9
-
3. Connecting to the Workstation
Before connecting for the first time, you should contact the administrators to ask for a username. You should also send your public key to the administrators.
-
3.1. VPN Connection Requirement (Off-Campus Access)
To access the GPU workstation from outside the ULB campus, you must first connect to the university’s VPN. A valid ULB NETID is required for authentication.
- Navigate to vpn.ulb.ac.be in your web browser.
- Log in using your ULB NETID and password.
- Download and install the VPN client software provided for your operating system.
- Launch the client and connect using the following details:
- Address: vpn.ulb.ac.be
- Login: Your NETID
Once connected, you will be able to access all protected ULB resources, including the GPU workstation.
The recommended client for Linux is openconnect.
- Install the client: bash sudo apt update && sudo apt install openconnect -y
- Connect to the VPN: bash sudo openconnect –protocol=gp vpn.ulb.ac.be You will be prompted to enter your NETID and password.
Important: You must keep this terminal window open for the duration of your session. Closing it will terminate the VPN connection.
-
3.2. Standard SSH Connection
You can connect to the workstation via SSH using its fully qualified domain name (recommended) or its IP address.
Using Hostname (Recommended):
ssh yourlogin@ulb-gpu.info.ulb.ac.be
Using IP Address:
ssh yourlogin@10.149.16.180
-
3.3. Streamlining Connections with an SSH Config File
For a more efficient workflow, you can configure an SSH client file to create aliases and automate connection settings like hostname, user, and port forwarding.
Create or edit the file at ~/.ssh/config and add an entry for the GPU server.
Example Configuration:
# GPU Workstation Alias
Host gpu-workstation
HostName ulb-gpu.info.ulb.ac.be
User your_login
Port 22
ForwardX11 yes
Compression yes
# Forward your assigned Jupyter port (e.g., 9120) to your local machine
LocalForward 9120 localhost:9120
# Example: Forward Spark UI port
LocalForward 4040 localhost:4040
How to Use: * With the configuration above, you can simply connect by typing: ssh gpu-workstation. * This configuration also works seamlessly with scp and sftp for file transfers. * LocalForward maps a port on your local machine to a port on the remote server, which is essential for accessing services like Jupyter notebooks in your local browser. * ForwardX11 yes enables GUI application forwarding, and Compression yes can speed up the connection over slower networks.
-
3.4. Port Redirection (Manual)
If you are not using an SSH config file, you can manually redirect ports using the -L flag. This example redirects port 8388 on the server to port 9999 on your local machine:
ssh -L 9999:localhost:8388 yourlogin@ulb-gpu.info.ulb.ac.be
-
- Processors: 2 x Intel Xeon E5-2640 V4 (10 cores, 2.4 GHz, up to 3.4 GHz Turbo)
- Motherboard: Asus Z10PG-D24 Server Board (Chipset Intel® C612 PCH)
- RAM: 24 x 32GB DDR4 ECC (Total 768GB) – Quad channel
- GPUs: 8 x Asus GeForce GTX 1080 Ti (Ref: TURBO-GTX1080TI-11G)
- 1 x 2TB NVMe SSD (boot drive)
- 6 x 2TB Seagate Enterprise HDDs (ST2000NX0403)
- 2 x Intel I210AT Gigabit ports
- Carte réseau PEB-10G/57840-2T – 2 x 10G LAN ports
- Power Supply: 3 x 1600W Redundant Power Supply
- Chassis: Boîtier ESC8000 G3 4U Cover
- Operating System: Ubuntu Desktop 22.04.4 LTS
- Linux Kernel: 5.15.0-113-generic x86_64
- NVIDIA Driver: 470.161.03
- CUDA Version: 11.4
- GCC Version: 9.4.0
New user creation requires a desired username (typically the netid) and a public SSH key.
Onboarding Commands:
username=”[NetID]”
sudo adduser $username
sudo usermod -aG docker $username
sudo chage -d 0 $username
sudo -u $username mkdir /home/$username/.ssh
echo “[SSH key]” | sudo -u $username tee /home/$username/.ssh/authorized_keys
The administrator will set a temporary password, which the user must change upon first login.
- nversbra: Nassim Versbraegen
- gbonte: Gianluca Bontempi
- gpaldino: Gian Marco Paldino
- dlunghi: Daniele Lunghi
- yamoling: Yannick Molinghen
- ptribel: Pascal Tribel (added 2023-10-14)
- amor0060: Alejandro Morales Hernández (added 2023-10-24)
- lcordeir: Loïc Cordeiro Fonseca (added 2025-02-16)
To maintain a stable environment and prevent configuration conflicts, all computations should be run inside a dedicated Docker container. Each user must build a personal clone of the base gpu_ubuntu1804 image.
-
6.1. Base Image Configuration
The base image includes pre-configured Python 3.6 and R 3.6.3 environments with common data science and deep learning libraries.
- Python: Includes Miniconda, Keras 2.4.0, TensorFlow 2.3, PyTorch, and libraries like pandas, scikit-learn, and matplotlib.
- R: Includes r-base, r-base-dev, and libraries like ggplot2, randomForest, and the IRKernel for Jupyter.
-
6.2. Building Your Personal Container
Create a Dockerfile with the following template. Obtain your userID and userGID by running the id command on the workstation.
FROM gpu_ubuntu1804:base
MAINTAINER Theo Verhelst <tverhels@ulb.ac.be>
ARG userPort=…
ARG userName=…
ARG userGID=…
ARG userID=…
# Create a non-root user for security
RUN groupadd -g $userGID $userName
RUN useradd -u $userID -d /home/$userName -ms /bin/bash -g $userGID -G sudo,$userName
USER $userName
WORKDIR /home/$userName
ENV PATH=/opt/miniconda3/bin:$PATH
# Create and set permissions for a shared data volume
RUN mkdir /home/$userName/shared_data
RUN chown $userName:$userName /home/$userName/shared_data
VOLUME /home/$userName/shared_data
EXPOSE $userPort
Build the image with this command, replacing $USER with your username:
nvidia-docker build –rm=true -t gpu_ubuntu1804:$USER .
-
6.3. Running Your Container
Use this command to launch your container in interactive mode:
nvidia-docker run –rm -it \
–gpus “device=#GPU_ID#” \
–memory=#Memory_limit# \
–memory-swap=#Swap_limit# \
-p #Jupyter_port#:#Jupyter_port# \
-v /path/to/host/data:/home/#userName#/shared_data \
gpu_ubuntu1804:$USER
Parameters: * #GPU_ID#: The numeric ID of the GPU allocated to you. * #Memory_limit#: The amount of RAM for the container (e.g., 16G). * #Swap_limit#: Total RAM + swap (e.g., 17G for 16G RAM and 1G swap). * #Jupyter_port#: Your assigned Jupyter port. * /path/to/host/data: An absolute path on the host machine to mount inside the container.
-
6.4. GPU and Jupyter Port Allocations
| GPU ID |
User(s) |
Jupyter Port(s) |
| 0 |
gbonte |
8890 |
| 1 |
jdestefa, cnachteg, ptribel |
9017, 9208, 9119 |
| 2 |
blebicho, ibosch |
9010 |
| 3 |
tverhels, jsal0013 |
8989, 9120 |
| 4 |
csimar |
– |
| 5, 6, 7 |
Reserved for experiments |
– |
| – |
yleborgn |
8889 |
| – |
blextrait |
9002 |
| – |
bvanderp |
8123 |
| – |
lcordeir |
9121 |
-
6.5. Optimizing GPU Memory Allocation (TensorFlow)
By default, TensorFlow pre-allocates all available GPU memory. To enable dynamic memory growth, add the appropriate code snippet to the beginning of your scripts.
Python:
import tensorflow as tf
gpus = tf.config.experimental.list_physical_devices(‘GPU’)
if gpus:
try:
for gpu in gpus:
tf.config.experimental.set_memory_growth(gpu, True)
except RuntimeError as e:
print(e)
-
7. Storage and Data Management
-
7.1. Disk Partitions and Mounts
Docker data resides on /media/hdd4. The other large disks are for general data storage.
| Device |
Mount point |
Filesystem |
Size |
| /dev/sda1 |
/media/hdd1 |
ext4 |
1.8T |
| /dev/sdb1 |
/media/hdd2 |
ext4 |
1.8T |
| /dev/sdc1 |
/media/hdd3 |
ext4 |
1.8T |
| /dev/sdd |
/media/hdd4 |
ext4 |
1.8T |
-
7.2. Accessing Large Datasets
For optimal performance, store large datasets on the /media/ drives rather than your home directory. Create a symbolic link for easy access:
# Example: Link a dataset on hdd1 to your home directory
ln -s /media/hdd1/my_large_dataset ~/datasets“`
## 8. Troubleshooting
### 8.1. SSH Connection Issues
**Error:** `ssh: Could not resolve hostname XXX: Name or service not known`
**Cause:** Your shell cannot find the `~/.ssh/config` file.
**Solution:** Specify the path to your config file manually using the `-F` flag:
“`bash
ssh -F ~/.ssh/config gpu-workstation
-
8.2. NVIDIA Driver Mismatch
Error Symptom: NVRM: API mismatch error in dmesg or /var/log/syslog.
Cause: The NVIDIA driver and kernel module versions are out of sync, typically after a system update.
Solution: 1. Update System: sudo apt update && sudo apt upgrade 2. Regenerate Initramfs: sudo update-initramfs -u -k all 3. Reboot: sudo reboot now
Error: “Got permission denied while trying to connect to the Docker daemon socket”
Solution: Your user account is not in the docker group. Contact an administrator to be added.
Error: “Error response from daemon: driver failed programming external connectivity”
Solution: The port you are trying to use is already allocated. Stop the existing container with nvidia-docker stop <container_name> or choose a different port.
Use these commands to check the status of system hardware:
- CPU Temperature: sensors (from the lm-sensors package)
- GPU Temperature and Usage: nvidia-smi
-
10. Maintenance Log Highlights
- 09/12/2022: Upgraded NVIDIA driver to 470.161.03 to resolve API mismatch.
- 08/12/2022: Removed a faulty RAM stick that was preventing boot.
- 12/2021: Upgraded total system RAM from 256GB to 768GB.
- 06/2021: Upgraded primary NVMe disk from 512GB to 2TB and installed Ubuntu 20.04.
