Cloud Networking

From CSCWiki
Revision as of 20:36, 29 December 2021 by Merenber (talk | contribs) (Created page with "This page gives an overview of the network topology of the CSC cloud. For the sections below, ".csc" is an abbreviation for ".csclub.uwaterloo.ca". == Hostnames and IP addre...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

This page gives an overview of the network topology of the CSC cloud.

For the sections below, ".csc" is an abbreviation for ".csclub.uwaterloo.ca".

Hostnames and IP addresses

Some important hostnames and IP addresses to know about:

  • VLAN 529 (172.19.168.0/27, fd74:6b6a:8eca:4902::/64) is the cloud management network
  • VLAN 425 (172.19.134.0/24, 2620:101:f000:4903::/64) is the cloud user network
  • Due to routing decisions, everything in VLAN 529 is accessible from VLAN 425, so make sure to use iptables if there's some service running in VLAN 529 which shouldn't be accessible by regular users. See the routers section below.
  • riboflavin, ginkgo and biloba are on VLAN 529 and VLAN 134
    • Each of these hosts has a ".cloud" domain record which resolves to their VLAN 529 address, e.g. riboflavin.cloud.csc
  • Most Ceph services are explicitly placed on riboflavin and ginkgo, since they have the most storage
    • ceph-nfs.cloud.csc and ceph-mon.cloud.csc are CNAMES for riboflavin and ginkgo
  • biloba and chamomile are the two CloudStack management servers
  • cloud.csc (and csclub.cloud) point to a virtual IP address (in VLAN 529) shared by biloba and chamomile via keepalived
    • CloudStack keeps cookie sessions in-memory, so it is important that only one machine is the "active" CloudStack management server at any time (otherwise, if you connect to a different machine, your cookie session will suddenly be invalid)
    • mariadb.cloud.csc points to this virtual IP address. We are using master-master replication between two MariaDB instances on biloba and chamomile. To avoid split-brain syndrome, it is very important that only instance gets written to at any time.
  • We are authoritative for the *.cloud.csc and *.csclub.cloud DNS zones, so do not use Infoblox for these (except for PTR records)
  • All *.csclub.cloud records point to the cloud.csc virtual IP

Cloud routers

The two cloud routers are router1.cloud.csc and router2.cloud.csc. They each have 3 IP addresses: one for each of VLANs 134, 425 and 529. router1.cloud.csc resolves to the VLAN 529 IP, and router1-cloud.csc resolves to the VLAN 134 IP (same for router2).

The two routers are also sharing the virtual IP addresses for router.cloud.csc (VLAN 529) and nat-gateway.cloud.csc (VLAN 425) via keepalived. For incoming VLAN 425 traffic, the routers basically strip off the VLAN 425 header and replace it with a VLAN 529 header, meaning that VLAN 529 is fully accessible from VLAN 425. Use iptables where necessary.

Setup

The routers are systemd-nspawn containers running on biloba and chamomile.

Retrieved from "https://wiki.csclub.uwaterloo.ca/index.php?title=Cloud_Networking&oldid=4657"