Friday, June 12, 2009

Check the device ASMLib on multi-path

It's just my curious with this forum on oracle. when you check multi-path device with ASMLib, perhaps you can confuse:

Example (Assume EMC Storage):
# /etc/init.d/oracleasm querydisk -d /dev/emcpowera1
Device "/dev/emcpowera1" is marked an ASM disk with the label "ARC1"
# /etc/init.d/oracleasm querydisk -d ARC1
Disk "ARC1" is a valid ASM disk on device [8, 38]
That show major,minor numbers -> 8, 38
So, check on device multi-path:
# ls -la /dev/emcpowera1
brw------- 1 root root 120, 1 Jun 11 17:01 /dev/emcpowera1
That made... confuse ;) because on /dev/emcpowera1 show major,minor numbers -> 120, 1

What's that mean? Because the ASMLib scanned on single path to show.
Really?
# ls -la /dev/sd* | grep 38
brw-rw---- 1 root disk 8, 38 Jun 11 17:01 /dev/sdc1
# powermt display dev=emcpowera
Pseudo name=emcpowera
Owner: default=SP A, current=SP A Array failover mode: 1
==============================================================================
---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
1 qla2xxx sdc SP A0 active alive 0 0
1 qla2xxx sde SP A1 active alive 0 0
1 qla2xxx sdg SP B0 active alive 0 0
1 qla2xxx sdi SP B1 active alive 0 0
That shows... single paths(sdc,sde,sdg,sdi) on /dev/emcpowera ;)

How we exclude scan on single paths? that's good question ... read on this site.
Excluding Single Path Disks

The system administrator configures ASMLib to ignore the single path disks. In the ASMLib configuration, he edits the ORACLEASM_SCANEXCLUDE variable to look like so:

ORACLEASM_SCANEXCLUDE="sdb sdc"

Here, the system administrator has been more specific. ASMLib should ignore exactly the disks /dev/sdb and /dev/sdc. It should not ignore other SCSI disks. While scanning, ASMLib will ignore those paths, only seeing the /dev/multipath disk. Once again, Oracle will use the multipath disk.
So, make it on example:

Edit /etc/sysconfig/oracleasm file.

# vi /etc/sysconfig/oracleasm
ORACLEASM_SCANEXCLUDE=""
change to =>
ORACLEASM_SCANEXCLUDE="sd"

And then restart:
#/etc/init.d/oracleasm restart

So, Check again:
# /etc/init.d/oracleasm querydisk /dev/emcpowera1
Device "/dev/emcpowera1" is marked an ASM disk with the label "ARC1"
# /etc/init.d/oracleasm querydisk -d ARC1
Disk "ARC1" is a valid ASM disk on device [120, 1]
# ls -al /dev/emcpowera1
brw------- 1 root root 120, 1 Jun 12 18:33 /dev/emcpowera1
On ASMLib show major,minor numbers -> 120, 1; that like on multi-path device (asmlib no scan on single paths) ;)

1 comment:

Fujisawa Rob said...

This helped me track down a problem where I could only mount a diskgroup on one node. The problem was that the node on which I had created the ASM disks was seeing them under the major/minor numbers of the /dev/mapper device. On the other nodes, when they scanned for ASM disks, they were seeing them as one of the single paths. I changed this behavior in /etc/sysconfig/oracleasm and it works great!