Proxmox

The Proxmox Vitural Environment (as of 2025-11-02) is a cluster hosted on citric-acid, tahini and teriyaki. The GUI can be accessed via the hostname of any machine on port 8006, e.g. https://citric-acid.csclub.uwaterloo.ca:8006.

Setting up Proxmox

To setup proxmox, from `Server View`, open the `Datacenter` page. Then go to `Permissions -> Realms`.

Then just make sure pam is setup lol

Joining the Cluster

To join the cluster, go to the existing CSC Cluster, from `Server View`, open the `Datacenter` page. Then go to `Cluster` and copy the join information.

On the new node, from `Server View`, open the `Datacenter` page. Then go to `Cluster`, then `Join Cluster` and paste the join information. You will need to enter the root password of the host where you copied the join information.

Ceph

We use Ceph to store VM and container disks as well as ISO images to allow easy migration between nodes.

Networking

There are two ways to do networking: network bridge and NAT. Network bridge will put the container/virtual machine on the CSC network (basically side-by-side to proxmox itself), while NAT will encapsulate the container/VM inside a private subnet that is only visible to proxmox host itself.

For services that only exposes HTTP/HTTPS, NAT is more desirable since multiple services can share a host nginx instance, only requiring the host IP to have 80/443 port opened to the Internet, thus saving some IP address in our pool and save some trips to the IST for firewall exemption. But for services that requires custom ports to be opened (for example, BigBlueButton requires a range of UDP ports to be exposed for relaying video streams), using the network bridge and giving the container/VM its own public IP might be easier.

Currently, vmbr0 is used for bridged network and vmbr1 is used for NAT (see Proxmox's wiki on NAT networking for setup instruction). vmbr0 uses the CSC DHCP server, so you can use DHCP there, but vmbr1 requires manual IP assignment.

Note that only using vmbr1 requires you to use SSH ProxyJump via citric-acid to access the inner container, as it wouldn't have a public IP.