Ceph - Revision history

Merenber at 11:03, 10 February 2024

2024-02-10T11:03:27Z

@@ Line 260: / Line 260: @@
 ceph orch daemon add osd ginkgo:/dev/sde
 </pre>
 == Miscellaneous commands ==

2023-03-29T15:35:14Z

Adding a new disk

← Older revision		Revision as of 10:35, 29 March 2023
Line 191:		Line 191:

	== Adding a new disk ==		== Adding a new disk ==
	Let's say we added a new disk /dev/sdg to ginkgo. Log in to one of the Ceph management servers (riboflavin or ginkgo)~~, run <code>cephadm shell</code> as root~~, then run		Let's say we added a new disk /dev/sdg to ginkgo. Log in to one of the Ceph management servers (riboflavin or ginkgo), then run
	<pre>		<pre>
	# clear any metadata at the start of the disk		# clear any metadata at the start of the disk
	dd if=/dev/zero of=/dev/sdg bs=1M count=10 conv=fsync		dd if=/dev/zero of=/dev/sdg bs=1M count=10 conv=fsync
			# Run this from inside `cephadm shell`
	ceph orch daemon add osd ginkgo:/dev/sdg		ceph orch daemon add osd ginkgo:/dev/sdg
	</pre>		</pre>

2023-03-28T22:11:13Z

Adding a new disk

← Older revision		Revision as of 17:11, 28 March 2023
Line 193:		Line 193:
	Let's say we added a new disk /dev/sdg to ginkgo. Log in to one of the Ceph management servers (riboflavin or ginkgo), run <code>cephadm shell</code> as root, then run		Let's say we added a new disk /dev/sdg to ginkgo. Log in to one of the Ceph management servers (riboflavin or ginkgo), run <code>cephadm shell</code> as root, then run
	<pre>		<pre>
			# clear any metadata at the start of the disk
			dd if=/dev/zero of=/dev/sdg bs=1M count=10 conv=fsync
	ceph orch daemon add osd ginkgo:/dev/sdg		ceph orch daemon add osd ginkgo:/dev/sdg
	</pre>		</pre>

2023-03-28T22:00:08Z

← Older revision		Revision as of 17:00, 28 March 2023
Line 189:		Line 189:
	</pre>		</pre>
	Now if you visit https://localhost:8443 (ignore the HTTPS warning), you can login to the dashboard. Credentials are stored in the usual place.		Now if you visit https://localhost:8443 (ignore the HTTPS warning), you can login to the dashboard. Credentials are stored in the usual place.

			== Adding a new disk ==
			Let's say we added a new disk /dev/sdg to ginkgo. Log in to one of the Ceph management servers (riboflavin or ginkgo), run <code>cephadm shell</code> as root, then run
			<pre>
			ceph orch daemon add osd ginkgo:/dev/sdg
			</pre>
			And that's it! You can run <code>ceph status</code> to see the progress of the PGs getting rebalanced.

	== Recovering from a disk failure ==		== Recovering from a disk failure ==

2023-03-28T21:53:02Z

Recovering from a disk failure

← Older revision		Revision as of 16:53, 28 March 2023
Line 247:		Line 247:
	After the drive has been replaced, zap it and add it to the cluster normally:		After the drive has been replaced, zap it and add it to the cluster normally:
	<pre>		<pre>
			dd if=/dev/zero of=/dev/sde bs=1M count=10 conv=fsync
	ceph-volume lvm zap /dev/sde
	ceph orch daemon add osd ginkgo:/dev/sde		ceph orch daemon add osd ginkgo:/dev/sde
	</pre>		</pre>

2023-03-26T13:36:00Z

Recovering from a disk failure

@@ Line 218: / Line 218: @@
 </pre>
 <pre>
 ceph osd out osd.3
 ceph orch daemon rm osd.3 --force
-ceph osd destroy osd.3 --yes-i-really-mean-it
+ceph osd purge osd.3 --yes-i-really-mean-it
 </pre>
-Now on the host with the disk, run:
+Destroy the LVM logical volume and volume group:
 <pre>
 </pre>

2023-03-21T22:27:04Z

Removing a disk

@@ Line 190: / Line 190: @@
 Now if you visit https://localhost:8443 (ignore the HTTPS warning), you can login to the dashboard. Credentials are stored in the usual place.
-== Removing a disk ==
+== Recovering from a disk failure ==
 <pre>
 ceph osd down osd.3

2021-12-29T20:29:45Z

Removing a disk

← Older revision		Revision as of 15:29, 29 December 2021
Line 210:		Line 210:
	vgdisplay		vgdisplay
	vgremove 4318d615-2cde-4ea1-a25a-9cba09821fc3		vgremove 4318d615-2cde-4ea1-a25a-9cba09821fc3
	# zero the beginning of the disk (~~make sure~~ you ~~have~~ ~~the~~ ~~right~~ ~~one!!~~)		# zero the beginning of the disk (if you plan on re-using it)
	dd if=/dev/zero of=/dev/sdc bs=1M count=10 conv=fsync		dd if=/dev/zero of=/dev/sdc bs=1M count=10 conv=fsync
	</pre>		</pre>

2021-12-29T20:28:25Z

@@ Line 189: / Line 189: @@
 </pre>
 Now if you visit https://localhost:8443 (ignore the HTTPS warning), you can login to the dashboard. Credentials are stored in the usual place.
 == Miscellaneous commands ==

2021-12-29T20:23:09Z

Miscellaneous commands

← Older revision		Revision as of 15:23, 29 December 2021
Line 191:		Line 191:

	== Miscellaneous commands ==		== Miscellaneous commands ==
	Here are some commands which may be useful:		Here are some commands which may be useful. See the [https://docs.ceph.com/en/latest/man/8/ceph/ man page] for a full reference.
	<ul>		<ul>
	<li>		<li>
Line 199:		Line 199:
	</pre>		</pre>
	Note: this doesn't actually show all of the individual disks. I think it might have to do with the hardware RAID controllers.		Note: this doesn't actually show all of the individual disks. I think it might have to do with the hardware RAID controllers.
			</li>
			<li>
			Show OSDs (Object Storage Daemons) on the current host (this needs to be run from <code>cephadm shell</code>):
			<pre>ceph-volume lvm list</pre>
	</li>		</li>
	<li>		<li>
	Show services:		Show services:
	<pre>ceph orch ls</pre>		<pre>ceph orch ls</pre>
			</li>
			<li>
			Show daemons of those services:
			<pre>ceph orch ps</pre>
			</li>
			<li>
			Show non-default config settings:
			<pre>ceph config dump</pre>
	</li>		</li>
	<li>		<li>
	Show pools:		Show pools:
	<pre>ceph osd pool ls detail</pre>		<pre>ceph osd pool ls detail</pre>
			</li>
			<li>
			List users:
			<pre>ceph auth ls</pre>
	</li>		</li>
	</ul>		</ul>