11gR2: setup a larger /dev/shm after ORA-00845: MEMORY_TARGET not supported on this system

If you get the following error



ORA-00845: MEMORY_TARGET not supported on this system

while using 11g and the AMM feature... it's time to increase your memory mapped files on /dev/shm.


As usually I've setup my /dev/shm devices at about half (4GB) of physical memory (8GB). I'm on a ProLiant DL180G5 and using OEL5.4. So the following was my file system configuration:
[root@rac31 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 267G 25G 228G 10% /
/dev/cciss/c0d0p1 99M 12M 82M 13% /boot
tmpfs 4.0G 192M 3.8G 5% /dev/shm
/dev/emcpowerb1 134G 1.1G 133G 1% /u02/ocfs2
/dev/emcpowerd1 5.0G 298M 4.8G 6% /u02/ocfs2_mirror


And my /etc/fstab:
[root@rac31 ~]# more /etc/fstab
LABEL=/1 / ext3 defaults 1 1
LABEL=/boot1 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SW-cciss/c0d0p3 swap swap defaults 0 0
/dev/emcpowerb1 /u02/ocfs2/ ocfs2 _netdev,datavolume,nointr 0 0
/dev/emcpowerd1 /u02/ocfs2_mirror/ ocfs2 _netdev,datavolume,nointr 0 0


You have to edit /etc/fstab and change the previous tmpfs line with the following one:
[root@rac31 ~]# vi /etc/fstab
LABEL=/1 / ext3 defaults 1 1
LABEL=/boot1 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs size=6g 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SW-cciss/c0d0p3 swap swap defaults 0 0
/dev/emcpowerb1 /u02/ocfs2/ ocfs2 _netdev,datavolume,nointr 0 0
/dev/emcpowerd1 /u02/ocfs2_mirror/ ocfs2 _netdev,datavolume,nointr 0 0


I then stopped CRS on only that node with the following command:
[root@rac31 ~]# /u01/app/11.2.0/grid/bin/crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac31'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac31'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac31'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac31'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac31'
CRS-2673: Attempting to stop 'ora.DATA_RAC3.dg' on 'rac31'
CRS-2673: Attempting to stop 'ora.FRA_RAC3.dg' on 'rac31'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac31'
CRS-2677: Stop of 'ora.DATA_RAC3.dg' on 'rac31' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.rac31.vip' on 'rac31'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac31'
CRS-2677: Stop of 'ora.rac31.vip' on 'rac31' succeeded
CRS-2672: Attempting to start 'ora.rac31.vip' on 'rac32'
CRS-2677: Stop of 'ora.scan1.vip' on 'rac31' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac32'
CRS-2677: Stop of 'ora.FRA_RAC3.dg' on 'rac31' succeeded
CRS-2677: Stop of 'ora.registry.acfs' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac31'
CRS-2677: Stop of 'ora.asm' on 'rac31' succeeded
CRS-2676: Start of 'ora.rac31.vip' on 'rac32' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac32' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac32'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.eons' on 'rac31'
CRS-2673: Attempting to stop 'ora.ons' on 'rac31'
CRS-2677: Stop of 'ora.ons' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac31'
CRS-2677: Stop of 'ora.net1.network' on 'rac31' succeeded
CRS-2677: Stop of 'ora.eons' on 'rac31' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac31' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac31'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac31'
CRS-2673: Attempting to stop 'ora.asm' on 'rac31'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac31'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac31'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac31'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac31'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac31' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac31' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac31'
CRS-2677: Stop of 'ora.cssd' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'rac31'
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac31'
CRS-2677: Stop of 'ora.gipcd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'rac31' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac31' has completed
CRS-4133: Oracle High Availability Services has been stopped.


No Oracle processes are now available.. I can simply reboot (yes I could mount it.. but I wanted to test it after a reboot)
[root@rac31 ~]# ps -ef|grep oracle
oracle 7079 1 0 Apr16 ? 00:00:01 /u01/app/oracle/product/11.2.0/dbhome_1/ccr/bin/nmz -cron -silent
root 28585 25068 0 17:42 pts/0 00:00:00 grep oracle

[root@rac31 ~]# reboot

Broadcast message from root (pts/0) (Tue Apr 27 17:42:42 2010):

The system is going down for reboot NOW!



And just few minutes after the instance (there was only the ASM one) started up I looked at my file system again.
[root@rac31 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 267G 25G 228G 10% /
/dev/cciss/c0d0p1 99M 12M 82M 13% /boot
tmpfs 6.0G 192M 5.9G 4% /dev/shm
/dev/emcpowerb1 134G 1.1G 133G 1% /u02/ocfs2
/dev/emcpowerd1 5.0G 295M 4.8G 6% /u02/ocfs2_mirror


Ok... it mounted my devices and it was now 6GB.

Now on the other node you have to repeat the same steps.
[root@rac32 ~]# vi /etc/fstab
LABEL=/ / ext3 defaults 1 1
LABEL=/boot /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs size=6g 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SW-cciss/c0d0p3 swap swap defaults 0 0
/dev/emcpowerb1 /u02/ocfs2/ ocfs2 _netdev,datavolume,nointr 0 0
/dev/emcpowerd1 /u02/ocfs2_mirror/ ocfs2 _netdev,datavolume,nointr 0 0


Stop the CRS.
[root@rac32 ~]# /u01/app/11.2.0/grid/bin/crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac32'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac32'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac32'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac32'
CRS-2673: Attempting to stop 'ora.DATA_RAC3.dg' on 'rac32'
CRS-2673: Attempting to stop 'ora.FRA_RAC3.dg' on 'rac32'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac32'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'rac32'
CRS-2677: Stop of 'ora.DATA_RAC3.dg' on 'rac32' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.rac32.vip' on 'rac32'
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rac32'
CRS-2677: Stop of 'ora.rac32.vip' on 'rac32' succeeded
CRS-2672: Attempting to start 'ora.rac32.vip' on 'rac31'
CRS-2677: Stop of 'ora.scan2.vip' on 'rac32' succeeded
CRS-2672: Attempting to start 'ora.scan2.vip' on 'rac31'
CRS-2677: Stop of 'ora.registry.acfs' on 'rac32' succeeded
CRS-2677: Stop of 'ora.FRA_RAC3.dg' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac32'
CRS-2677: Stop of 'ora.asm' on 'rac32' succeeded
CRS-2676: Start of 'ora.rac32.vip' on 'rac31' succeeded
CRS-2676: Start of 'ora.scan2.vip' on 'rac31' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'rac31'
CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac32'
CRS-2673: Attempting to stop 'ora.eons' on 'rac32'
CRS-2677: Stop of 'ora.ons' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac32'
CRS-2677: Stop of 'ora.net1.network' on 'rac32' succeeded
CRS-2677: Stop of 'ora.eons' on 'rac32' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac32' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac32'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac32'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac32'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac32'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac32'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac32'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac32' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac32' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac32' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac32' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac32' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac32'
CRS-2677: Stop of 'ora.cssd' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac32'
CRS-2673: Attempting to stop 'ora.diskmon' on 'rac32'
CRS-2677: Stop of 'ora.gpnpd' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac32'
CRS-2677: Stop of 'ora.gipcd' on 'rac32' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'rac32' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac32' has completed
CRS-4133: Oracle High Availability Services has been stopped.


After a reboot the ASM instance was running again


and my /dev/shm devie was 6GB.
[root@rac32 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 267G 16G 238G 7% /
/dev/cciss/c0d0p1 99M 12M 82M 13% /boot
tmpfs 6.0G 192M 5.9G 4% /dev/shm
/dev/emcpowerb1 134G 1.1G 133G 1% /u02/ocfs2
/dev/emcpowerd1 5.0G 293M 4.8G 6% /u02/ocfs2_mirror


That's all.

11gR2: how to remove file from ASM diskgroup

I needed to delete some asm files from my previous database installation.


I wanted to use ASMCMD tool so I loaded my ASM environment:
[oracle@rac32 ~]$ . oraenv
ORACLE_SID = [RAC32] ? +ASM2
The Oracle base for ORACLE_HOME=/u01/app/11.2.0/grid is /u01/app/oracle


Then I entered into asmcms tool:
[oracle@rac32 ~]$ asmcmd
ASMCMD> ls
DATA_RAC3/
FRA_RAC3/
ASMCMD> exit
[oracle@nov2211 ~]$ asmcmd -p
ASMCMD [+] > cd DATA_RAC3
ASMCMD [+DATA_RAC3] > ls
RAC3/
ASMCMD [+DATA_RAC3] > cd RAC3
ASMCMD [+DATA_RAC3/RAC3] > ls
ARCHIVELOG/
ASMPARAMETERFILE/
OCRFILE/

Ok .. there are some files and I would like to remove them.
ASMCMD [+DATA_RAC3/RAC3] > rm ARCHIVE*
You may delete multiple files and/or directories.
Are you sure? (y/n) y
ORA-15032: not all alterations performed
ORA-15177: cannot operate on system aliases (DBD ERROR: OCIStmtExecute)

That error could be tied at the metalink note 444812.1 and bug 3751057.8


In that doc id they suggest to run a sql script and recreate the alter diskgroup drop file syntax.
Likely I found the following link (and thanks to Freek D'Hooge) http://www.dba-village.com/village/dvp_scripts.ScriptDetails?ScriptIdA=2403#
that help me to reach my goal.
SQL> column a format a120
SQL> set linesize 120
SQL> select 'alter diskgroup ' || gname || ' drop file ''+' || gname || sys_connect_by_path(aname, '/') || ''';' a
2 from ( select b.name gname, a.parent_index pindex, a.name aname,
3 a.reference_index rindex , a.system_created, a.alias_directory
4 from v$asm_alias a, v$asm_diskgroup b
5 where a.group_number = b.group_number
6 )
7 where alias_directory = 'N'
8 and system_created = 'Y'
9 start with (mod(pindex, power(2, 24))) = 0
10 and rindex in
11 ( select a.reference_index
12 from v$asm_alias a, v$asm_diskgroup b
13 where a.group_number = b.group_number
14 and (mod(a.parent_index, power(2, 24))) = 0
15 and a.name = '&DATABASENAME'
16 )
17 connect by prior rindex = pindex;
Enter value for databasename: RAC3
old 15: and a.name = '&DATABASENAME'
new 15: and a.name = 'RAC3'

A
------------------------------------------------------------------------------------------------------------------------
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ASMPARAMETERFILE/REGISTRY.253.706985453';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/OCRFILE/REGISTRY.255.706985457';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_1_seq_81.272.707781651';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_2_seq_30.271.707775999';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_1_seq_80.270.707775999';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_1_seq_82.273.707787599';
...
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_699.1067.714573049';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_700.1058.714573053';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_701.1066.714573059';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_2_seq_579.1065.714573061';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_702.1064.714573069';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_691.1452.714532055';

1176 rows selected.



Then I simply copied and executed the output:
SQL> alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_1_seq_81.272.707781651';

Diskgroup altered.

[oracle@nov2211 ~]$ asmcmd -p
ASMCMD [+] > lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 511993 511882 0 511882 0 N DATA_RAC3/
MOUNTED EXTERN N 512 4096 1048576 511993 511896 0 511896 0 N FRA_RAC3/





That's all

11gR2: adding a mirrored voting disk on OCFS2 file system

I want to add a mirrored voting disk after migration from ASM to OCFS2.
The following is my current configuration:
/u01/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 353efdf9e50d4f93bf71c6aa84b9e19b (/u02/ocfs2/css/vdisk) []
Located 1 voting disk(s).


Create a new directory on mirrored location


Set the right permissions, as I did:
touch css/vdisk
chown -R oracle.oinstall css/

and check if all is right

Now execute the command to add another voting disk:
/u01/app/11.2.0/grid/bin/crsctl add css votedisk /u02/ocfs2_mirror/css/vdisk
Now formatting voting disk: /u02/ocfs2_mirror/css/vdisk.
CRS-4603: Successful addition of voting disk /u02/ocfs2_mirror/css/vdisk.

As you can see, I have now two voting disk files.
/u01/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 353efdf9e50d4f93bf71c6aa84b9e19b (/u02/ocfs2/css/vdisk) []
2. ONLINE 0c254a5cdcd14f9bbfef2866447e908c (/u02/ocfs2_mirror/css/vdisk) []
Located 2 voting disk(s).


But what about if I want to have a mirrored copy also on Oracle ASM ?
It's simply not possible even if from the documentation I didn't find about it.
/u01/app/11.2.0/grid/bin/crsctl add css votedisk +DATA_RAC3
CRS-4671: This command is not supported for ASM diskgroups.
CRS-4000: Command Add failed, or completed with errors.

The command is not supported. I can mention this documentation note from "Clusterware Administration and Deployment Guide" (e10717.pdf):
Oracle does not support storing the OCR on different storage types simultaneously, such as storing OCR on both Oracle ASM and a shared file system, except during a migration.


That's all.

11gR2: removing a mirrored Oracle Cluster Registry and adding again

I want to test a scenario where one of my OCR file is lost.

This is what I have done. Now let's have a look at my OCR configuration.

/u01/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2708
Available space (kbytes) : 259412
ID : 2044210075
Device/File Name : /u02/ocfs2/ocr/OCRFile
Device/File integrity check succeeded
Device/File Name : /u02/ocfs2_mirror/ocr/OCRFile
Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

As you can see I've two mirrored OCR files one on /u02/ocfs2/ path and another on /u02/ocfs2_mirror/. For your information, these paths are on two different LUNs formed by several disks and on a RAID-1 configuration

Here is where my mirrored OCR file exists.

Now I delete it

Issuing:
/u01/app/11.2.0/grid/bin/ocrcheck
PROT-602: Failed to retrieve data from the cluster registry
PROC-26: Error while accessing the physical storage

it says it's no more able to get data from OCR, even if my cluster is still up.

Now I try to recreate the "lost" OCR mirrored file

At first I simply tried to add the new OCR file, but I got:
/u01/app/11.2.0/grid/bin/ocrconfig -add /u02/ocfs2_mirror/ocr/OCRFile
PROT-29: The Oracle Cluster Registry location is already configured

Clear... I have to first remove from the configuration the "old" location of lost OCR file and then add it again.
/u01/app/11.2.0/grid/bin/ocrconfig -delete /u02/ocfs2_mirror/ocr/OCRFile
/u01/app/11.2.0/grid/bin/ocrconfig -add /u02/ocfs2_mirror/ocr/OCRFile

As you can see my OCR configuration is mirrored again:
/u01/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2708
Available space (kbytes) : 259412
ID : 2044210075
Device/File Name : /u02/ocfs2/ocr/OCRFile
Device/File integrity check succeeded
Device/File Name : /u02/ocfs2_mirror/ocr/OCRFile
Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

That's all.

11gR2: Migrating Oracle Cluster Registry from Oracle ASM to Other Types of Storage

The following are the steps to migrate OCR from Oracle ASM to OCFS2. According to "Clusterware Administration and Deployment Guide" (e10717.pdf) run /u01/app/11.2.0/grid/bin/ocrcheck to see the status of cluster registry. Then create a file (just touch it) under the preferred directory and set the right permissions. This is what I did as root user:

touch /u02/ocfs2/ocr/OCRFile
chown root.oinstall /u02/ocfs2/ocr/OCRFile
chmod 640 /u02/ocfs2/ocr/OCRFile

Then add your new OCR file to the final directory:

/u01/app/11.2.0/grid/bin/ocrconfig -add /u02/ocfs2/ocr/OCRFile

and check again the OCR status:

/uo1/app/11.2.0./grid/bin/ocrcheck

Now it's time to remove OCR file from ASM.

Nothing to do on second node.
That's all.

11gR2: Migrating Voting Disks from Oracle ASM to Other Types of Storage

The following are the steps to migrate voting disk from Oracle ASM to OCFS2. During these steps I found a problem already documented on metalink as a documentation bug, that is documentation is missing some informations.

Run

/u01/app/11.2.0/grid/bin/crsctl query css votedisk

to see the status and location of voting disk (in the following image it's the second command displayed just after ocrcheck command)


Create on the shared OCFS2 file system a directory where relocate the voting disk and set the right permissions. I've created ocr (useful for me for relocate also OCR file on OCFS2 file system) and css directories, owned by oracle:

chown -R oracle.oinstall css


Now it's time to run the following command (on the image below just look at the second command issued. On the first one I've missed to write the voting disk file name):

/u01/app/11.2.0/grid/bin/crsctl replace votedisk /u02/ocfs2/css/vdisk
but I received the following errors:
CRS-4256: Updating the profile
Failed to update the profile with the new discovery string
CRS-4000: Command Replace failed, or completed with errors.



And indeed the voting disk is still on Oracle ASM


Under css directory, a file was created. Having a look on permissions granted on that vdisk file... we can see it's owned by oracle.root.

Delete that file, create a new empty one (just touch it) and grant the following permissions:

chown oracle.oinstall vdisk

Using the previous command it's finally able to replace the voting disk.


Now you can see that the new voting disk is relocated on file system


That's all.