1. Installing and setting Ubuntu 20.04

Windows OS is no good, in my opinion, for doing any ML development or networking work. Therefore, we will set-up Linux for that purpose. Ubuntu Desktop 20.04 (download here) is an ideal choice for that, as a lot of functionally works out-of-the-box, allowing us to save on the set-up time as compared to the other Linux distributions. You have three options to proceed:

  1. Install Ubuntu only.
  2. Dual-booting Ubuntu alongside an existing Windows OS. Follow the instruction here.
  3. Use WSL2 (Windows Subsystem for Linux); this is discussed here. Warning: I have never tested this option (you are on your own here!).

I recommend option #2, to preserve access to Windows (for gaming-related purposes). Make sure to connect your machine to the internet during the installation to get the latest updates and drivers.

After the installation, login and update and install necessary packages, e.g.

sudo apt-get update &&
sudo apt-get -y upgrade &&
sudo apt-get -y install build-essential gcc g++ make binutils &&
sudo apt-get -y install software-properties-common git &&
sudo apt-get install build-essential cmake git pkg-config

2. Setting remote access

In this section, we will set-up a secure way to remotely log in to the machine. Option to access the GPU machine from any network is also discussed.

ssh

For now, login into Ubuntu, open the terminal and install the ssh-server:

sudo apt update &&
sudo apt install openssh-server

After that, verify the ssh-server was installed correctly

sudo systemctl status ssh

and open the firewall

sudo ufw allow ssh

Test your ssh connection by running, from your laptop (which is assumed to be connected to the same network as your GPU machine)

ssh user@<local-ip-address>

where user is your Ubuntu username, while your local IP address on Ubuntu (e.g. 172.148.0.14) can be found with ip add, yielding (e.g.)

link/ether b3:21:88:6k:17:24 brd ff:ff:ff:ff:ff:ff
inet 172.148.0.14/32 scope global noprefixroute enp7s3

Make a note of your MAC address (e.g. b3:21:88:6k:17:24) and network card name (e.g. enp7s3), for later.

Next, set-up ssh keys for password-less login as described here. After that, assuming you will only ever want to login to your machine from this laptop, disable the option to log in with a password altogether. This will only allow for ssh connections from devices with ssh-key already exchanged, improving on security. Edit /etc/ssh/sshd_config (with sudo-privileges) on Ubuntu to add

#only ssh-key logins 
PermitRootLogin no
ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no
AllowUsers user
PubkeyAuthentication yes
# keep ssh connection alive
ClientAliveInterval 60
TCPKeepAlive yes
ClientAliveCountMax 10000

Don’t forget to restart ssh-service for it to pick-up on the changes with

sudo restart ssh

DNS

If you want to ssh to your machine from outside your home network, you will need to establish a permanent DNS ‘hostname’ that is independent of your machine’s global IP addresses, which your ISP can change at any time. I have personally used free service from no-IP (link here, to get started use “Create Your Free Hostname Now” from Ubuntu’s browser); alternative DNS services exist like FreeDNS, Dynu etc. Be creative with your hostname; I cannot believe mlgpu.ddns.net is not yet taken. This hostname will be used as an example in this article.

After that, install the Dynamic Update Client (DUC) as described here. This ensures that when your global IP does change, it is re-linked to your hostname (e.g. mlgpu.ddns.net). To ensure DUC is started every time with Ubuntu, add a new cronjob with sudo crontab -e (i.e. add this line to the end of the file),

@reboot su -l user && sudo noip2

and reboot your machine.

Port-forwarding

ssh by default uses port 22 for the connection. We need to open this port (for TCP and UDP packets) through the router to be able to ssh to the machine from an outside network (you still want to submit those GPU-intensive jobs from a coffee shop, right?). This setting is usually found under “Advanced/Security” tab. Below is a screenshot of my router settings

Both UDP and TCP protocols are allowed through port 22 of Ubuntu’s network card.

Optional: While digging through your router settings, add your Ubuntu’s network card MAC address and local IP to the reserved list (‘Add reserved rule’), and save your router settings to a file (in case you need to reload this at a later date).

Finally, on your laptop, edit ~/.ssh/config replacing user with your Ubuntu’s user name, and HostName with your DNS hostname

# keep ssh connection alive
TCPKeepAlive yes
ServerAliveInterval 60
# assign a 'shortcut' for hostname and enable graphical display over ssh (X11)
Host mlgpu
HostName mlgpu.ddns.net
GSSAPIAuthentication yes
User user
ForwardX11Trusted yes
ForwardX11 yes
GSSAPIDelegateCredentials yes

This will allow you ssh to your machine quickly and test that the DNS and port-forwarding are working with

ssh mlgpu

This should NOT ask you for the password, and you can now ssh to your GPU machine from any network!

Wake-on-LAN (WAL)

Imagine a scenario where you are at a conference (before 2020 those took place at a remote location other than your home — bizarre, I know!) and you want to train your DNN on your GPU machine. But wait, you turned it off before leaving your house — oh dear!

This is where WAL and magic packets come in!

On Ubuntu, edit /etc/systemd/system/wol@.service

[Unit]
Description=Wake-on-LAN for %i
Requires=network.target
After=network.target
[Service]
ExecStart=/sbin/ethtool -s %i wol g
Type=oneshot
[Install]
WantedBy=multi-user.target

After that, enable WOL service via

sudo systemctl enable wol@enp7s3 && 
systemctl is-enabled wol@np7s3

This should return ‘enabled’; enp7s3 is the network card name returned with ip add command. Finally, check your BIOS setting for “wake-on-lan” = enabled.

Now, on your laptop, install wakeonlan via

brew install wakeonlan

To test WOL is working, shutdown Ubuntu and issue the following command from your laptop using your DNS hostname (or IP address) and MAC address

wakeonlan -i mlgpu.ddns.net -p 22 b3:21:88:6k:17:24

This should wake-up your GPU-machine from sleep or powered-off state (make sure it’s still connected to power, of course!). If you went for the dual-boot option, and Ubuntu is not the first OS in the boot menu, adjust this with the GRUB.