CloudStack - Revision history

K57chan at 19:55, 1 July 2025

2025-07-01T19:55:27Z

← Older revision		Revision as of 14:55, 1 July 2025
Line 435:		Line 435:
	# you need to reload both nginx & cloudstack to get it running		# you need to reload both nginx & cloudstack to get it running
	# You need the crt to get it working (stored under `/etc/csc/k8s` (I think)		# You need the crt to get it working (stored under `/etc/csc/k8s` (I think)

	#
			== Crash Out Notes Cont. ==
			The user console on CloudStack was not working.

			In Chamomile <code>/etc/nginx/sites-enabled</code>, you can see a list of all user websites that are enabled. There's a file entitled <code>consoleproxy.cloud.csclub.uwaterloo.ca</code> in which you will find that the console proxy is being hosted on the VM with IP <code>172.19.134.59</code> on CloudStack.

			Run the following command whilst in Chamomile to access the console proxy:

			<code>sudo ssh -i /var/lib/cloudstack/management/.ssh/id_rsa -p 3922 169.254.35.150</code>

			More information on troubleshooting the console and console proxy can be found [https://cwiki.apache.org/confluence/display/CLOUDSTACK/View+Console+and+Console+Proxy+Troubleshooting here.]

S23adhik at 06:59, 15 June 2025

2025-06-15T06:59:08Z

← Older revision		Revision as of 01:59, 15 June 2025
Line 425:		Line 425:

	If you decide to perform an upgrade, then at the very least, you will need to backup the MariaDB databases ('cloud' and 'cloud_usage'), as well as the /etc/cloudstack and /var/lib/cloudstack folders on each of biloba, chamomile and ginkgo. Also, good luck.		If you decide to perform an upgrade, then at the very least, you will need to backup the MariaDB databases ('cloud' and 'cloud_usage'), as well as the /etc/cloudstack and /var/lib/cloudstack folders on each of biloba, chamomile and ginkgo. Also, good luck.

			== Siracha Crash Out Notes ==
			If you ever find your self in the unfourtunate position of needing to recover from a drive failure. It's JOEVER BRO TRUST ME.

			Anyways this is what I've learned from the pits of hell

			# Our nginx config is funny, and it redirects 9987 → gunicorn → ceod instance running on server
			## If the drive dies, ask someone with a backup for just a zip file, it's not possible to recover. Trust me
			# you need to reload both nginx & cloudstack to get it running
			# You need the crt to get it working (stored under `/etc/csc/k8s` (I think)
			#

Merenber at 01:34, 19 September 2024

2024-09-19T01:34:40Z

@@ Line 152: / Line 152: @@
 dupload cloudstack_4.16.0.0+1_amd64.changes
 </pre>
 == Database setup ==

Merenber: /* Some more global settings */

2023-05-15T22:29:05Z

Some more global settings

← Older revision		Revision as of 17:29, 15 May 2023
Line 369:		Line 369:
	max.account.secondary.storage = 20		max.account.secondary.storage = 20
	</pre>		</pre>

			<b>NOTE</b>: the <code>cpu.overprovisioning.factor</code> setting also needs to be set for existing clusters. Go to Infrastructure -> Clusters -> Cluster1 -> Settings and set it accordingly.

	=== Firewall ===		=== Firewall ===

Merenber: /* Unexpected reboot */

2023-01-27T04:47:01Z

Unexpected reboot

← Older revision		Revision as of 23:47, 26 January 2023
Line 27:		Line 27:

	== Unexpected reboot ==		== Unexpected reboot ==
	Sometimes a network interface fails on a machine after the switches in MC are rebooted (looking at you, riboflavin). Or a machine randomly goes offline in the middle of the night (looking at you, ginkgo). Point is, sometimes a machine needs to rebooted, or is forcefully rebooted, without preparation. Unfortunately, <strong>CloudStack is unable to recover gracefully from an unexpected reboot</strong>. This means that <strong>manual intervention~~</strong>~~ is required to get the VMs back into a working state.		Sometimes a network interface fails on a machine after the switches in MC are rebooted (looking at you, riboflavin). Or a machine randomly goes offline in the middle of the night (looking at you, ginkgo). Point is, sometimes a machine needs to rebooted, or is forcefully rebooted, without preparation. Unfortunately, <strong>CloudStack is unable to recover gracefully from an unexpected reboot</strong>. This means that <strong>manual intervention is required</strong> to get the VMs back into a working state.

	Once the machine has come back online, perform the following:		Once the machine has come back online, perform the following:

Merenber at 04:46, 27 January 2023

2023-01-27T04:46:25Z

@@ Line 25: / Line 25: @@
 <br><br>
 Once all VMs have been migrated, do whatever you need to do on the physical host; once it is back up, take it back out of maintenance mode from the web UI. Repeat for any other hosts which need to be taken offline.
 == Administration ==

Merenber at 04:36, 27 January 2023

2023-01-27T04:36:24Z

@@ Line 6: / Line 6: @@
 * [[Cloud Networking]]
-Official CloudStack documentation: http://docs.cloudstack.apache.org/en/latest/
+Official CloudStack documentation: http://docs.cloudstack.apache.org/en/4.16.0.0/
 == Rebooting machines ==
@@ Line 25: / Line 25: @@
 <br><br>
 Once all VMs have been migrated, do whatever you need to do on the physical host; once it is back up, take it back out of maintenance mode from the web UI. Repeat for any other hosts which need to be taken offline.
 == Building packages ==

Merenber at 01:47, 16 June 2022

2022-06-16T01:47:45Z

← Older revision		Revision as of 20:47, 15 June 2022
Line 9:		Line 9:

	== Rebooting machines ==		== Rebooting machines ==
	I'm going to start with this first because this is what future sysadmins are most interested in. If you reboot one of the CloudStack guest machines (as of this writing: biloba, ginkgo and chamomile), then I suggest you perform a live migration of all of the VMs on that host to the other other machines, ~~then~~ ~~move them back after the~~ reboot.		I'm going to start with this first because this is what future sysadmins are most interested in. If you reboot one of the CloudStack guest machines (as of this writing: biloba, ginkgo and chamomile), then I suggest you perform a live migration of all of the VMs on that host to the other other machines (see [[#Sequential reboot]]).

	If this is not possible (e.g. there is not enough capacity on the other machines), then CloudStack will most likely shut down the VMs automatically. <b>You are responsible for restarting them manually after the reboot.</b> You will also need to manually restart any Kubernetes clusters.		If this is not possible (e.g. there is not enough capacity on the other machines), then CloudStack will most likely shut down the VMs automatically. <b>You are responsible for restarting them manually after the reboot.</b> You will also need to manually restart any Kubernetes clusters.

	Note: if the cloudstack-agent.service is having trouble reconnecting to the management servers after a reboot, just do a systemctl restart and cross your fingers.		Note: if the cloudstack-agent.service is having trouble reconnecting to the management servers after a reboot, just do a systemctl restart and cross your fingers.

			=== Sequential reboot ===
			If it is possible to reboot the machines one at a time (e.g. for a software upgrade), then it is possible to avoid having any downtime. Login to the web UI as admin, go to Infrastructure > Hosts, hover above the three-dots button for a particular host, then press the "Enable Maintenance Mode" button.
			[[File:Cloudstack-enable-maintenance-mode-button.png\|1000px]]
			<br><br>
			Wait for the VMs to be migrated to the other machines (press the Refresh button to update the table). If you see an error which says "ErrorInPrepareForMaintenance", just wait it out. If more than 20 minutes have passed and there is still no progress, take the host out of maintenance mode, and put it back into maintenance mode. If this still does not work, restart the management server.

			When a host is in maintenance mode, it should look like this:
			[[File:Cloudstack-host-in-maintenance-mode.png\|1000px]]
			<br><br>
			Once all VMs have been migrated, do whatever you need to do on the physical host; once it is back up, take it back out of maintenance mode from the web UI. Repeat for any other hosts which need to be taken offline.

	== Building packages ==		== Building packages ==

Merenber: /* Rebooting machines */

2022-05-13T19:39:19Z

Rebooting machines

← Older revision		Revision as of 14:39, 13 May 2022
Line 12:		Line 12:

	If this is not possible (e.g. there is not enough capacity on the other machines), then CloudStack will most likely shut down the VMs automatically. <b>You are responsible for restarting them manually after the reboot.</b> You will also need to manually restart any Kubernetes clusters.		If this is not possible (e.g. there is not enough capacity on the other machines), then CloudStack will most likely shut down the VMs automatically. <b>You are responsible for restarting them manually after the reboot.</b> You will also need to manually restart any Kubernetes clusters.

			Note: if the cloudstack-agent.service is having trouble reconnecting to the management servers after a reboot, just do a systemctl restart and cross your fingers.

	== Building packages ==		== Building packages ==

Merenber at 19:38, 13 May 2022

2022-05-13T19:38:14Z

← Older revision		Revision as of 14:38, 13 May 2022
Line 7:		Line 7:

	Official CloudStack documentation: http://docs.cloudstack.apache.org/en/latest/		Official CloudStack documentation: http://docs.cloudstack.apache.org/en/latest/

			== Rebooting machines ==
			I'm going to start with this first because this is what future sysadmins are most interested in. If you reboot one of the CloudStack guest machines (as of this writing: biloba, ginkgo and chamomile), then I suggest you perform a live migration of all of the VMs on that host to the other other machines, then move them back after the reboot.

			If this is not possible (e.g. there is not enough capacity on the other machines), then CloudStack will most likely shut down the VMs automatically. <b>You are responsible for restarting them manually after the reboot.</b> You will also need to manually restart any Kubernetes clusters.

	== Building packages ==		== Building packages ==