Enabling ASM Filter Driver

As promised, this entry documents my experience in enabling the ASM Filter Driver (ASMFD). The ASMFD is the replace for ASM that according to Oracle prevents non-oracle software from making modifications to Oracle database files. Upgrading from standard ASM was surprisingly difficult, and required much more time than expected. Most of the problems arose from the difficulty in ensuring that all of the original ASM software was disabled during the process of moving to ASMFD. Also causing issues was the requirement to periodically run commands as root or grid, depending on the command. It is very important to run the commands as the correct user id.

Note that the example here is the more complex ASM case of the OCR and Voting Disks being stored in ASM.

So, here is the process.
1. Install the patch for Oracle bug 19035573. This fixes several issues with the upgrade process. Note, the readme for this patch installation is incomplete. Review the Oracle notes for patch installation using opatch prior to installing the patch. I required several tries to get this to work right. In particular, you must run the unlock perl script.
in the Grid Infrastructure home prior to patching, and the post patch script when complete.

2. Identify the OS path of all of the Oracle ASM disks. Oracle Support supplied me with this script which extracted the path for my Oracle disks. Note that this example was on a VM, and I was using shared VMDK’s for this process. Also note that ASM was already properly configured, and the cluster was up and running when this process was started (root user).

/etc/init.d/oracleasm querydisk -d `/etc/init.d/oracleasm listdisks -d` | 
cut -f2,10,11 -d" " | perl -pe 's/"(.*)".*\[(.*), *(.*)\]/$1 $2 $3/g;' | 
while read v_asmdisk v_minor v_major 
do 
v_device=`ls -la /dev | grep " $v_minor, *$v_major " | awk '{print $10}'` 
echo "ASM disk $v_asmdisk based on /dev/$v_device [$v_minor, $v_major]" 
done

Output from above:
ASM disk DATA based on /dev/sdf1 [8, 81]
ASM disk OCR1 based on /dev/sdc1 [8, 33]
ASM disk OCR2 based on /dev/sdd1 [8, 49]
ASM disk OCR3 based on /dev/sde1 [8, 65]
ASM disk RECO based on /dev/sdg1 [8, 97]

3. Add AFD:* to the asm_diskstring so that the asm_diskstring contains both the old and new path NOTE: This is run as the GI home owner (GRID):

$ORACLE_HOME/bin/asmcmd dsset ‘ORCL:*’,’AFD:*’

4. Stop the cluster (run as root):
crsctl stop crs –f (both nodes)

5. Stop asm (as root):
/etc/init.d/oracleasm stop
/etc/init.d/oracleasm disable
Move the file /opt/oracle/extapi/64/asm/orcl/1/libasm.so to a new location so it is out of your path.

6. Stop ACFS (as root):
$ORACLE_HOME/bin/acfsload stop

7. At this point, if not already installed, oracle 12.1 GI should be installed.

8. On both nodes, configure the afd driver:
$ORACLE_HOME/bin/asmcmd afd_configure

9. Label the ASM disks for AFD (as root).
afd_label OCR1 /dev/sdc1 –migrate
afd_label OCR2 /dev/sdd1 –migrate
afd_label OCR3 /dev/sde1 –migrate
afd_label DATA /dev/sdf1 –migrate
afd_label RECO /dev/sdg1 –migrate

10. Scan for the new disks (as GRID):
$ORACLE_HOME/bin/asmcmd dsget
$ORACLE_HOME/bin/asmcmd afd_scan

11. Check the directory /dev/oracleafd/disks/* and verify that the ASMFD disks are listed there.

12. Start the cluster. If possible, boot the server to verify that everything starts properly after boot.

A couple of notes on this process: It took me several days to get this right, and I had to open an SR with Oracle support along the way. Evidently it is common for the ASM drivers to remain loaded, as well as the ACFS drivers. When you get error messages about ACFS or ASM being loaded, you may need to take additional steps to make sure that they are unloaded.
Also, the instructions from Oracle support said to set the afd diskstring to /dev/sd* as the last step before the scan. This may or may not be necessary, I did not run the command, and the upgrade process ran properly. I suspect when using the ASM_DISKSTRING as ORCL:* and the AFD_DISKSTRING as AFD:* this is not needed.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: