11gR2: setup a larger /dev/shm after ORA-00845: MEMORY_TARGET not supported on this system

If you get the following error



ORA-00845: MEMORY_TARGET not supported on this system

while using 11g and the AMM feature... it's time to increase your memory mapped files on /dev/shm.


As usually I've setup my /dev/shm devices at about half (4GB) of physical memory (8GB). I'm on a ProLiant DL180G5 and using OEL5.4. So the following was my file system configuration:
[root@rac31 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 267G 25G 228G 10% /
/dev/cciss/c0d0p1 99M 12M 82M 13% /boot
tmpfs 4.0G 192M 3.8G 5% /dev/shm
/dev/emcpowerb1 134G 1.1G 133G 1% /u02/ocfs2
/dev/emcpowerd1 5.0G 298M 4.8G 6% /u02/ocfs2_mirror


And my /etc/fstab:
[root@rac31 ~]# more /etc/fstab
LABEL=/1 / ext3 defaults 1 1
LABEL=/boot1 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SW-cciss/c0d0p3 swap swap defaults 0 0
/dev/emcpowerb1 /u02/ocfs2/ ocfs2 _netdev,datavolume,nointr 0 0
/dev/emcpowerd1 /u02/ocfs2_mirror/ ocfs2 _netdev,datavolume,nointr 0 0


You have to edit /etc/fstab and change the previous tmpfs line with the following one:
[root@rac31 ~]# vi /etc/fstab
LABEL=/1 / ext3 defaults 1 1
LABEL=/boot1 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs size=6g 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SW-cciss/c0d0p3 swap swap defaults 0 0
/dev/emcpowerb1 /u02/ocfs2/ ocfs2 _netdev,datavolume,nointr 0 0
/dev/emcpowerd1 /u02/ocfs2_mirror/ ocfs2 _netdev,datavolume,nointr 0 0


I then stopped CRS on only that node with the following command:
[root@rac31 ~]# /u01/app/11.2.0/grid/bin/crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac31'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac31'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac31'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac31'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac31'
CRS-2673: Attempting to stop 'ora.DATA_RAC3.dg' on 'rac31'
CRS-2673: Attempting to stop 'ora.FRA_RAC3.dg' on 'rac31'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac31'
CRS-2677: Stop of 'ora.DATA_RAC3.dg' on 'rac31' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.rac31.vip' on 'rac31'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac31'
CRS-2677: Stop of 'ora.rac31.vip' on 'rac31' succeeded
CRS-2672: Attempting to start 'ora.rac31.vip' on 'rac32'
CRS-2677: Stop of 'ora.scan1.vip' on 'rac31' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac32'
CRS-2677: Stop of 'ora.FRA_RAC3.dg' on 'rac31' succeeded
CRS-2677: Stop of 'ora.registry.acfs' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac31'
CRS-2677: Stop of 'ora.asm' on 'rac31' succeeded
CRS-2676: Start of 'ora.rac31.vip' on 'rac32' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac32' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac32'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.eons' on 'rac31'
CRS-2673: Attempting to stop 'ora.ons' on 'rac31'
CRS-2677: Stop of 'ora.ons' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac31'
CRS-2677: Stop of 'ora.net1.network' on 'rac31' succeeded
CRS-2677: Stop of 'ora.eons' on 'rac31' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac31' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac31'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac31'
CRS-2673: Attempting to stop 'ora.asm' on 'rac31'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac31'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac31'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac31'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac31'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac31' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac31' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac31'
CRS-2677: Stop of 'ora.cssd' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'rac31'
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac31'
CRS-2677: Stop of 'ora.gipcd' on 'rac31' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'rac31' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac31' has completed
CRS-4133: Oracle High Availability Services has been stopped.


No Oracle processes are now available.. I can simply reboot (yes I could mount it.. but I wanted to test it after a reboot)
[root@rac31 ~]# ps -ef|grep oracle
oracle 7079 1 0 Apr16 ? 00:00:01 /u01/app/oracle/product/11.2.0/dbhome_1/ccr/bin/nmz -cron -silent
root 28585 25068 0 17:42 pts/0 00:00:00 grep oracle

[root@rac31 ~]# reboot

Broadcast message from root (pts/0) (Tue Apr 27 17:42:42 2010):

The system is going down for reboot NOW!



And just few minutes after the instance (there was only the ASM one) started up I looked at my file system again.
[root@rac31 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 267G 25G 228G 10% /
/dev/cciss/c0d0p1 99M 12M 82M 13% /boot
tmpfs 6.0G 192M 5.9G 4% /dev/shm
/dev/emcpowerb1 134G 1.1G 133G 1% /u02/ocfs2
/dev/emcpowerd1 5.0G 295M 4.8G 6% /u02/ocfs2_mirror


Ok... it mounted my devices and it was now 6GB.

Now on the other node you have to repeat the same steps.
[root@rac32 ~]# vi /etc/fstab
LABEL=/ / ext3 defaults 1 1
LABEL=/boot /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs size=6g 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SW-cciss/c0d0p3 swap swap defaults 0 0
/dev/emcpowerb1 /u02/ocfs2/ ocfs2 _netdev,datavolume,nointr 0 0
/dev/emcpowerd1 /u02/ocfs2_mirror/ ocfs2 _netdev,datavolume,nointr 0 0


Stop the CRS.
[root@rac32 ~]# /u01/app/11.2.0/grid/bin/crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac32'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac32'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac32'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac32'
CRS-2673: Attempting to stop 'ora.DATA_RAC3.dg' on 'rac32'
CRS-2673: Attempting to stop 'ora.FRA_RAC3.dg' on 'rac32'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac32'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'rac32'
CRS-2677: Stop of 'ora.DATA_RAC3.dg' on 'rac32' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.rac32.vip' on 'rac32'
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rac32'
CRS-2677: Stop of 'ora.rac32.vip' on 'rac32' succeeded
CRS-2672: Attempting to start 'ora.rac32.vip' on 'rac31'
CRS-2677: Stop of 'ora.scan2.vip' on 'rac32' succeeded
CRS-2672: Attempting to start 'ora.scan2.vip' on 'rac31'
CRS-2677: Stop of 'ora.registry.acfs' on 'rac32' succeeded
CRS-2677: Stop of 'ora.FRA_RAC3.dg' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac32'
CRS-2677: Stop of 'ora.asm' on 'rac32' succeeded
CRS-2676: Start of 'ora.rac32.vip' on 'rac31' succeeded
CRS-2676: Start of 'ora.scan2.vip' on 'rac31' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'rac31'
CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'rac31' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac32'
CRS-2673: Attempting to stop 'ora.eons' on 'rac32'
CRS-2677: Stop of 'ora.ons' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac32'
CRS-2677: Stop of 'ora.net1.network' on 'rac32' succeeded
CRS-2677: Stop of 'ora.eons' on 'rac32' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac32' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac32'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac32'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac32'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac32'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac32'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac32'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac32' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac32' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac32' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac32' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac32' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac32'
CRS-2677: Stop of 'ora.cssd' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac32'
CRS-2673: Attempting to stop 'ora.diskmon' on 'rac32'
CRS-2677: Stop of 'ora.gpnpd' on 'rac32' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac32'
CRS-2677: Stop of 'ora.gipcd' on 'rac32' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'rac32' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac32' has completed
CRS-4133: Oracle High Availability Services has been stopped.


After a reboot the ASM instance was running again


and my /dev/shm devie was 6GB.
[root@rac32 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 267G 16G 238G 7% /
/dev/cciss/c0d0p1 99M 12M 82M 13% /boot
tmpfs 6.0G 192M 5.9G 4% /dev/shm
/dev/emcpowerb1 134G 1.1G 133G 1% /u02/ocfs2
/dev/emcpowerd1 5.0G 293M 4.8G 6% /u02/ocfs2_mirror


That's all.

11gR2: how to remove file from ASM diskgroup

I needed to delete some asm files from my previous database installation.


I wanted to use ASMCMD tool so I loaded my ASM environment:
[oracle@rac32 ~]$ . oraenv
ORACLE_SID = [RAC32] ? +ASM2
The Oracle base for ORACLE_HOME=/u01/app/11.2.0/grid is /u01/app/oracle


Then I entered into asmcms tool:
[oracle@rac32 ~]$ asmcmd
ASMCMD> ls
DATA_RAC3/
FRA_RAC3/
ASMCMD> exit
[oracle@nov2211 ~]$ asmcmd -p
ASMCMD [+] > cd DATA_RAC3
ASMCMD [+DATA_RAC3] > ls
RAC3/
ASMCMD [+DATA_RAC3] > cd RAC3
ASMCMD [+DATA_RAC3/RAC3] > ls
ARCHIVELOG/
ASMPARAMETERFILE/
OCRFILE/

Ok .. there are some files and I would like to remove them.
ASMCMD [+DATA_RAC3/RAC3] > rm ARCHIVE*
You may delete multiple files and/or directories.
Are you sure? (y/n) y
ORA-15032: not all alterations performed
ORA-15177: cannot operate on system aliases (DBD ERROR: OCIStmtExecute)

That error could be tied at the metalink note 444812.1 and bug 3751057.8


In that doc id they suggest to run a sql script and recreate the alter diskgroup drop file syntax.
Likely I found the following link (and thanks to Freek D'Hooge) http://www.dba-village.com/village/dvp_scripts.ScriptDetails?ScriptIdA=2403#
that help me to reach my goal.
SQL> column a format a120
SQL> set linesize 120
SQL> select 'alter diskgroup ' || gname || ' drop file ''+' || gname || sys_connect_by_path(aname, '/') || ''';' a
2 from ( select b.name gname, a.parent_index pindex, a.name aname,
3 a.reference_index rindex , a.system_created, a.alias_directory
4 from v$asm_alias a, v$asm_diskgroup b
5 where a.group_number = b.group_number
6 )
7 where alias_directory = 'N'
8 and system_created = 'Y'
9 start with (mod(pindex, power(2, 24))) = 0
10 and rindex in
11 ( select a.reference_index
12 from v$asm_alias a, v$asm_diskgroup b
13 where a.group_number = b.group_number
14 and (mod(a.parent_index, power(2, 24))) = 0
15 and a.name = '&DATABASENAME'
16 )
17 connect by prior rindex = pindex;
Enter value for databasename: RAC3
old 15: and a.name = '&DATABASENAME'
new 15: and a.name = 'RAC3'

A
------------------------------------------------------------------------------------------------------------------------
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ASMPARAMETERFILE/REGISTRY.253.706985453';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/OCRFILE/REGISTRY.255.706985457';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_1_seq_81.272.707781651';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_2_seq_30.271.707775999';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_1_seq_80.270.707775999';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_1_seq_82.273.707787599';
...
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_699.1067.714573049';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_700.1058.714573053';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_701.1066.714573059';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_2_seq_579.1065.714573061';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_702.1064.714573069';
alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_03_25/thread_1_seq_691.1452.714532055';

1176 rows selected.



Then I simply copied and executed the output:
SQL> alter diskgroup DATA_RAC3 drop file '+DATA_RAC3/RAC3/ARCHIVELOG/2010_01_08/thread_1_seq_81.272.707781651';

Diskgroup altered.

[oracle@nov2211 ~]$ asmcmd -p
ASMCMD [+] > lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 511993 511882 0 511882 0 N DATA_RAC3/
MOUNTED EXTERN N 512 4096 1048576 511993 511896 0 511896 0 N FRA_RAC3/





That's all