NetApp - Revision history

Ehashman: /* Terminology */ jxpryde nitpicking me

2015-12-17T02:52:31Z

Terminology: jxpryde nitpicking me

← Older revision		Revision as of 21:52, 16 December 2015
Line 15:		Line 15:
	* '''Disk shelf:''' where the physical disks live. Can be plugged into a filer or directly into another machine.		* '''Disk shelf:''' where the physical disks live. Can be plugged into a filer or directly into another machine.
	* '''RAID:''' "Redundant Array of Independent Disks", used to improve reliability and protect against disk failures.		* '''RAID:''' "Redundant Array of Independent Disks", used to improve reliability and protect against disk failures.
	* '''RAID-DP:''' "Double Parity" RAID, similar to RAID6 (~~this~~ ~~is probably~~ RAID4 with two parity disks~~, but the result is the same~~). It ~~uses two parity disks and~~ can survive up to two disk failures before degradation.		* '''RAID-DP:''' "Double Parity" RAID, similar to RAID6 in failure tolerance (implemented like RAID4 but with two dedicated parity disks). It can survive up to two disk failures before degradation.
	* '''aggr:''' An aggregate of disks. This is a list of physical disks, similar to selecting the physical devices used for LVM.		* '''aggr:''' An aggregate of disks. This is a list of physical disks, similar to selecting the physical devices used for LVM.
	* '''vol:''' A volume consisting of some space on an aggregate. In general, we use the whole aggregate for a volume. RAID level is set at the volume. Similar to an LVM volume group.		* '''vol:''' A volume consisting of some space on an aggregate. In general, we use the whole aggregate for a volume. RAID level is set at the volume. Similar to an LVM volume group.

Ehashman: /* NetApp Configuration */ Add expanding LUN section

2015-12-17T02:49:44Z

NetApp Configuration: Add expanding LUN section

← Older revision		Revision as of 21:49, 16 December 2015
Line 134:		Line 134:

	You're done! Any host in the initiator group should now be able to access the LUN you've created as a block device.		You're done! Any host in the initiator group should now be able to access the LUN you've created as a block device.

			===Expanding an aggregate, volume, and LUN===

			1. Start by getting the aggregate's status, e.g.

			psilodump> '''aggr status -r aggr3'''
			Aggregate aggr3 (online, raid_dp) (block checksums)
			Plex /aggr3/plex0 (online, normal, active)
			RAID group /aggr3/plex0/rg0 (normal)

			RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
			--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
			dparity 0c.32 0c 2 0 FC:B - FCAL 10000 136000/278528000 139072/284820800
			parity 0c.33 0c 2 1 FC:B - FCAL 10000 136000/278528000 139072/284820800
			data 0a.34 0a 2 2 FC:A - FCAL 10000 136000/278528000 139072/284820800
			...

			2. Now determine the available spare disks:
			psilodump> '''aggr status -s'''

			Spare disks

			RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
			--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
			Spare disks for block or zoned checksum traditional volumes or aggregates
			spare 0a.41 0a 2 9 FC:A - FCAL 10000 136000/278528000 137104/280790184
			spare 0c.38 0c 2 6 FC:B - FCAL 10000 136000/278528000 137104/280790184
			spare 0c.37 0c 2 5 FC:B - FCAL 10000 136000/278528000 137422/281442144
			...

			3. Select disks by device number and add them to the aggregate, using the following command. (Use the -n flag if you want to test your command syntax with a dry run.)

			psilodump> '''aggr add aggr3 -g rg0 -d 0a.39 0a.44 0c.40 0c.45'''
			Addition of 4 disks to the aggregate has completed.
			Wed Dec 16 19:55:09 EST [psilodump: raid.vol.disk.add.done:notice]: Addition of Disk /aggr3/plex0/rg0/0c.45 Shelf 2 Bay 13 [NETAPP X274_HJURE146F10 NA14] S/N [404W6272] to aggregate aggr3 has completed successfully
			...

			4. Now fight with `vol size` to resize the volume:

			psilodump> '''df -A aggr3'''
			Aggregate kbytes used avail capacity
			aggr3 833369408 357122492 476246916 43%
			psilodump> '''vol size vol3backup +476246000k'''
			vol size: Insufficient space to grow this volume with its guarantee enabled; maximum growth is +473602692k.
			psilodump> '''vol size vol3backup +473602692k'''
			vol size: Flexible volume 'vol3backup' size set to 828725892k.

			5. Last, fight with `lun resize` to increase the lun size:

			psilodump> '''lun resize /vol/vol3backup/lun0 +473602692k'''
			lun resize: No space left on device
			lun resize: max size: 788g (846844657664)
			psilodump> '''lun resize /vol/vol3backup/lun0 846844657664'''

	==Host Configuration==		==Host Configuration==

Jxpryde: Raid correction

2015-12-17T02:45:59Z

Raid correction

← Older revision		Revision as of 21:45, 16 December 2015
Line 15:		Line 15:
	* '''Disk shelf:''' where the physical disks live. Can be plugged into a filer or directly into another machine.		* '''Disk shelf:''' where the physical disks live. Can be plugged into a filer or directly into another machine.
	* '''RAID:''' "Redundant Array of Independent Disks", used to improve reliability and protect against disk failures.		* '''RAID:''' "Redundant Array of Independent Disks", used to improve reliability and protect against disk failures.
	* '''RAID-DP:''' "Double Parity" RAID, similar to RAID6. It uses two parity disks and can survive up to two disk failures before degradation.		* '''RAID-DP:''' "Double Parity" RAID, similar to RAID6 (this is probably RAID4 with two parity disks, but the result is the same). It uses two parity disks and can survive up to two disk failures before degradation.
	* '''aggr:''' An aggregate of disks. This is a list of physical disks, similar to selecting the physical devices used for LVM.		* '''aggr:''' An aggregate of disks. This is a list of physical disks, similar to selecting the physical devices used for LVM.
	* '''vol:''' A volume consisting of some space on an aggregate. In general, we use the whole aggregate for a volume. RAID level is set at the volume. Similar to an LVM volume group.		* '''vol:''' A volume consisting of some space on an aggregate. In general, we use the whole aggregate for a volume. RAID level is set at the volume. Similar to an LVM volume group.

Ehashman: Major updates to the page; re-ordered it for ease of navigation

2015-12-17T02:09:57Z

Major updates to the page; re-ordered it for ease of navigation

Show changes

Ehashman: /* Disk information */

2015-12-17T01:49:58Z

Disk information

← Older revision		Revision as of 20:49, 16 December 2015
Line 222:		Line 222:
	* aggr1		* aggr1
	** Music aggregate volume, in RAID-DP		** Music aggregate volume, in RAID-DP
	* aggr_users
⚫	** ~~Users aggregate~~ volume, in RAID-DP
	* aggr2		* aggr2
	** ~~Old users~~ aggregate volume, in RAID-DP		** Users aggregate volume, in RAID-DP
			* aggr3
		⚫	** Backups volume for CSC videos, in RAID-DP

	===Volumes===		===Volumes===
Line 234:		Line 234:
	* /vol/vol2users		* /vol/vol2users
	** Users volume. This volume is not accessible via NFS or CIFS. It contains only the iSCSI LUN /vol/vol2users/lun0 .		** Users volume. This volume is not accessible via NFS or CIFS. It contains only the iSCSI LUN /vol/vol2users/lun0 .
			* /vol/vol3backup
			** Backup volume for videos. This volume is not accessible via NFS or CIFS. It contains only the iSCSI LUN /vol/vol3backup/lun0 .

	==Commands==		==Commands==

Ehashman: /* Configuration */ re-ordered and clarified some directions

2015-07-22T23:09:45Z

Configuration: re-ordered and clarified some directions

← Older revision		Revision as of 18:09, 22 July 2015
Line 22:		Line 22:

	This section describes how to create a volume on the NetApp and export it as an iSCSI target. For further NetApp configuration instructions, refer to the NetApp documentation.		This section describes how to create a volume on the NetApp and export it as an iSCSI target. For further NetApp configuration instructions, refer to the NetApp documentation.

			====One-time Configuration====

		⚫	Enable iSCSI and configure default authentication.

		⚫	options iscsi.enable on
		⚫	iscsi nodename iqn.1992-08.com.netapp:psilodump.csclub.uwaterloo.ca
		⚫	iscsi security default -s CHAP -p yoursecurepassword -n psilodump

		⚫	where yoursecurepassword is more secure. For iSCSI hosts, the target will be on node iqn.1992-08.com.netapp:psilodump.csclub.uwaterloo.ca with username psilodump and password yoursecurepassword.

			====Setting up a new disk aggregate, volume, and LUN====

	1. Login to the NetApp. You'll either need access to the physical serial console or to ssh as root to psilodump's private IP (10.15.134.130). Credentials are stored in /users/sysadmin .		1. Login to the NetApp. You'll either need access to the physical serial console or to ssh as root to psilodump's private IP (10.15.134.130). Credentials are stored in /users/sysadmin .
Line 43:		Line 55:
	vol create volNfoo -s volume aggrN XXXK		vol create volNfoo -s volume aggrN XXXK

	where XXX is the total available space in aggrN. You may need to choose a ~~slightly~~ smaller number due to hidden size constraints and rounding.		where XXX is the total available space in aggrN. You may need to choose a smaller number due to hidden size constraints and rounding. If you can't seem to find the right size, pick one much smaller, and then use the command

			vol size volNfoo +XXX

			to grow the volume. This command will tell you how much available space remains, unlike `vol create`, so you don't need to keep guessing.

	7. Disable snapshotting and access time update. Neither will be needed for exporting an iSCSI LUN.		7. Disable snapshotting and access time update. Neither will be needed for exporting an iSCSI LUN.
Line 50:		Line 66:
	snap reserve volNfoo 0		snap reserve volNfoo 0

		⚫	8. Create a LUN on your volume:
⚫	8. Enable iSCSI and configure default authentication.
⚫	options iscsi.enable on
⚫	iscsi nodename iqn.1992-08.com.netapp:psilodump.csclub.uwaterloo.ca
⚫	iscsi security default -s CHAP -p yoursecurepassword -n psilodump

⚫	where yoursecurepassword is more secure. For iSCSI hosts, the target will be on node iqn.1992-08.com.netapp:psilodump.csclub.uwaterloo.ca with username psilodump and password yoursecurepassword.

⚫	9. Create a LUN on your volume:
	lun create -s XXXK -t linux /vol/volNfoo/lun0		lun create -s XXXK -t linux /vol/volNfoo/lun0

	where XXXK is the amount of available space on the volume, as shown by the command df.		where XXXK is the amount of available space on the volume, as shown by the command df.

	10. Create an iSCSI initiator group and add all of your hosts to it:		9. Create an iSCSI initiator group and add all of your hosts to it:
	igroup create -i -t linux volNfoo_group		igroup create -i -t linux volNfoo_group
	igroup add volNfoo_group iqn.1993-08.org.debian:01:123456789		igroup add volNfoo_group iqn.1993-08.org.debian:01:123456789
Line 70:		Line 79:
	The node identifiers given to the igroup add command will soon be able to access the iSCSI LUN you created above.		The node identifiers given to the igroup add command will soon be able to access the iSCSI LUN you created above.

	11. Map the LUN to the iSCSI initiator group:		10. Map the LUN to the iSCSI initiator group:
	lun map /vol/volNfoo/lun0 volNfoo_group		lun map /vol/volNfoo/lun0 volNfoo_group

Ehashman: /* Terminology */ this is wrong

2015-07-16T03:08:10Z

Terminology: this is wrong

← Older revision		Revision as of 22:08, 15 July 2015
Line 239:		Line 239:

	==Terminology==		==Terminology==
	* RAID-DP - Double Parity ~~RAID6~~		* RAID-DP - Double Parity RAID4

M4burns at 17:59, 18 May 2014

2014-05-18T17:59:02Z

← Older revision		Revision as of 12:59, 18 May 2014
Line 14:		Line 14:
	Configuration mechanisms are accessible either via SSH or serial interface, but through aspartame only. The NetApp is not visible on 134net at all.		Configuration mechanisms are accessible either via SSH or serial interface, but through aspartame only. The NetApp is not visible on 134net at all.

			The private IP is 10.15.134.130, only available from aspartame on the interface with IP 10.15.134.1. You may have to remove the default route from the routing table in order to successfully contact the machine with ssh.
	Private IP TBD.

	==Configuration==		==Configuration==
Line 196:		Line 196:
	==Disk information==		==Disk information==
	* shelf 1		* shelf 1
	** ~~14x???~~ 10,000RPM FibreChannel disks		** 14x136GB 10,000RPM FibreChannel disks
	** Currently ~~set~~ to ~~standalone~~ ~~filer+shelf,~~ ~~not~~ ~~set~~ up		** Currently disconnected, could be connected to psilodump or directly to another machine.
	* shelf 2		* shelf 2
	** ~~14x???~~ 10,000RPM FibreChannel disks		** 14x136GB 10,000RPM FibreChannel disks
	** Currently assigned to ~~phlogiston, not set up (phlogiston is off)~~		** Currently assigned to psilodump
	* shelf 3		* shelf 3
	** 14x500GB 7,200RPM ATA disks		** 14x500GB 7,200RPM ATA disks
Line 213:		Line 213:
	* aggr1		* aggr1
	** Music aggregate volume, in RAID-DP		** Music aggregate volume, in RAID-DP
			* aggr_users
⚫	* aggr2
	** Users aggregate volume, in RAID-DP		** Users aggregate volume, in RAID-DP
		⚫	* aggr2
			** Old users aggregate volume, in RAID-DP

	===Volumes===		===Volumes===

M4burns: /* Other notes */

2013-02-04T22:06:13Z

Other notes

← Older revision		Revision as of 17:06, 4 February 2013
Line 154:		Line 154:
	==Other notes==		==Other notes==

	~~4. Set up the netapp filesystem and transfer~~ old files from ginseng:		===Transferring old files from ginseng===

	===Method A===		====Method A====

	* On ginseng, use parted to set up the mounted iscsi drive as an ext4 primary partition (setting up a partition of size >2TB requires care and a GPT)		* On ginseng, use parted to set up the mounted iscsi drive as an ext4 primary partition (setting up a partition of size >2TB requires care and a GPT)
Line 170:		Line 170:
	-C /export/users $(notdir $<) /mnt/iscsi		-C /export/users $(notdir $<) /mnt/iscsi

	===Method B===		====Method B====

	* On ginseng, authenticate with iSCSI target (psilodump.csclub.uwaterloo.ca lun0).		* On ginseng, authenticate with iSCSI target (psilodump.csclub.uwaterloo.ca lun0).
Line 179:		Line 179:
	resize2fs /path/to/psilodump:lun0		resize2fs /path/to/psilodump:lun0

	~~5. '''~~Exporting Kerberized NFS from Debian Sid~~'''~~		===Exporting Kerberized NFS from Debian Sid===

	The default kernel in Debian sid (stable, 2.6.32) does not support the necessary crypto suites to export kerberized NFS to newer kernels. You MUST upgrade the kernel, nfs-common, and nfs-kernel-server packages to AT LEAST squeeze-backports.		The default kernel in Debian sid (stable, 2.6.32) does not support the necessary crypto suites to export kerberized NFS to newer kernels. You MUST upgrade the kernel, nfs-common, and nfs-kernel-server packages to AT LEAST squeeze-backports.

			===iSCSI block device mount optimizations===

			tmyklebu made some changes to /sys/block/sda/queue. The following is now in /etc/rc.local on aspartame:

			echo 2048 > /sys/block/sda/queue/read_ahead_kb
			echo 32768 > /sys/block/sda/queue/max_sectors_kb
			echo 4096 > /sys/block/sda/queue/nr_requests
			echo noop > /sys/block/sda/queue/scheduler

			We should increase the iSCSI configs node.session.queue_depth and node.session.cmds_max during next maintenance window.

	==Disk information==		==Disk information==

M4burns: /* NetApp Configuration */

2013-02-04T18:18:33Z

NetApp Configuration

← Older revision		Revision as of 13:18, 4 February 2013
Line 70:		Line 70:
	The node identifiers given to the igroup add command will soon be able to access the iSCSI LUN you created above.		The node identifiers given to the igroup add command will soon be able to access the iSCSI LUN you created above.

	11 Map the LUN to the iSCSI initiator group:		11. Map the LUN to the iSCSI initiator group:
	lun map /vol/volNfoo/lun0 volNfoo_group		lun map /vol/volNfoo/lun0 volNfoo_group