To setup rda collection at cluster

Read more…

Read more…

While installing ASM instance I came across Ora 29701

Reason : Css daemon is not running

Solution: cd $ORACLE_HOME/bin

localconfig delete
localconfig add

For more details visit metalink note 259301.1 & download showcrs shell script
[oracle@ora02 ~]$ crs_stat -t
Name Type Target State Host
————————————————————
ora….SM1.asm application ONLINE ONLINE ora01
ora….01.lsnr application ONLINE ONLINE ora01
ora….a01.gsd application ONLINE UNKNOWN ora01
ora….a01.ons application ONLINE UNKNOWN ora01
ora….a01.vip application ONLINE ONLINE ora01
ora….SM2.asm application ONLINE ONLINE ora02
ora….02.lsnr application ONLINE ONLINE ora02
ora….a02.gsd application ONLINE UNKNOWN ora02
ora….a02.ons application ONLINE UNKNOWN ora02
ora….a02.vip application ONLINE ONLINE ora02
ora.prod.db application ONLINE ONLINE ora01
ora….prod.cs application OFFLINE OFFLINE
ora….od1.srv application ONLINE UNKNOWN ora01
ora….od2.srv application ONLINE UNKNOWN ora02
ora….d1.inst application ONLINE ONLINE ora01
ora….d2.inst application ONLINE ONLINE ora02

[oracle@ora02 ~]$ srvctl config nodeapps -n ora02
ora02 prod2 /u01/app/oracle/product/10.2.0/db_1
Check VIP [oracle@ora02 ~]$ srvctl config nodeapps -n ora02 -a
VIP exists.: /oravip02.wyg-asp.com/10.13.100.14/255.255.255.0/bond0
Check GSD [oracle@ora02 ~]$ srvctl config nodeapps -n ora02 -g
GSD exists.
Check ONS [oracle@ora02 ~]$ srvctl config nodeapps -n ora02 -s
ONS daemon exists.
Check Listener [oracle@ora02 ~]$ srvctl config nodeapps -n ora02 -l
Listener exists.

[oracle@ora01 bin]$ srvctl start nodeapps -n ora01
CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource ‘ora.ora01.gsd’ has placement error.
CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource ‘ora.ora01.ons’ has placement error.
[oracle@ora01 bin]$ crs_stop ora.ora01.gsd -f
Attempting to stop `ora.ora01.gsd` on member `ora01`
Stop of `ora.ora01.gsd` on member `ora01` succeeded.
[oracle@ora01 bin]$ crs_stop ora.ora01.ons -f
Attempting to stop `ora.ora01.ons` on member `ora01`
Stop of `ora.ora01.ons` on member `ora01` succeeded.

To clear the state of the resource you have to use the force option in the crs_stop command

[oracle@ora01 bin]$ crs_stop ora.ora02.ons -f
Attempting to stop `ora.ora02.ons` on member `ora02`
Stop of `ora.ora02.ons` on member `ora02` succeeded.
[oracle@ora01 bin]$ crs_stop ora.ora02.gsd -f
Attempting to stop `ora.ora02.gsd` on member `ora02`
Stop of `ora.ora02.gsd` on member `ora02` succeeded.
[oracle@ora01 bin]$ srvctl start nodeapps -n ora02

[oracle@ora01 bin]$ srvctl start nodeapps -n ora01

[oracle@ora01 bin]$ srvctl start nodeapps -n ora02
[oracle@ora01 bin]$ showcrs
HA Resource Target State
———– —— —–
ora.ora01.ASM1.asm ONLINE ONLINE on ora01
ora.ora01.LISTENER_ora01.lsnr ONLINE ONLINE on ora01
ora.ora01.gsd ONLINE ONLINE on ora01
ora.ora01.ons ONLINE ONLINE on ora01
ora.ora01.vip ONLINE ONLINE on ora01
ora.ora02.ASM2.asm ONLINE ONLINE on ora02
ora.ora02.LISTENER_ora02.lsnr ONLINE ONLINE on ora02
ora.ora02.gsd ONLINE ONLINE on ora02
ora.ora02.ons ONLINE ONLINE on ora02
ora.ora02.vip ONLINE ONLINE on ora02
ora.prod.db ONLINE ONLINE on ora01
ora.prod.prod.cs OFFLINE OFFLINE
ora.prod.prod.prod1.srv ONLINE UNKNOWN on ora01
ora.prod.prod.prod2.srv ONLINE UNKNOWN on ora02
ora.prod.prod1.inst ONLINE ONLINE on ora01
ora.prod.prod2.inst ONLINE ONLINE on ora02

I could get ASM commandline to discover ASM disks at both RAC nodes. It can map VOL1, VOL2 at both nodes but Oracle RAC database installer won’t see ASM disks.

Operating System Linux Red Hat Advanced Server OS Version 4.0 Update Version Nahant Update 3
On WYGORA01
[root@wygora01 software]# /etc/init.d/oracleasm createdisk VOL1 /dev/mapper/mpath5
Marking disk “/dev/mapper/mpath5” as an ASM disk: [ OK ]
[root@wygora01 software]# /etc/init.d/oracleasm createdisk VOL1 /dev/mapper/mpath6
root@wygora01 software]# /etc/init.d/oracleasm createdisk VOL2 /dev/mapper/mpath6
Marking disk “/dev/mapper/mpath6” as an ASM disk: [ OK ]
On WYGORA02
[root@wygora02 ~]# /etc/init.d/oracleasm scandisks
Scanning system for ASM disks: [ OK ]

[root@wygora02 ~]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
[root@wygora01 ~]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
Initially I was trying to get oracle installer to create ASM instance. Somehow installer couldn’t list VOL1 & VOL2 disks though ASM listdisk could see it. After carrying ASM dropdisk/createdisk fixed it but restart was causing same issue. I decided to go ahead with installation and see if Oracle builds an environment . Installer managed to start ASM1 instance but failed to build ASM2 instance on another RAC node.

I tried starting the ASM2 instance manually and came across error

SQL> startup pfile=/u01/app/oracle/product/10.2.0/asm/admin/+ASM/pfile/init.ora
ASM instance started
Total System Global Area 130023424 bytes
Fixed Size 2019032 bytes
Variable Size 102838568 bytes
ASM Cache 25165824 bytes
ORA-15032: not all alterations performed
ORA-15063: ASM discovered an insufficient number of disks for diskgroup “DATA”
SQL> select PATH, MOUNT_STATUS, HEADER_STATUS, MODE_STATUS, STATE from V$ASM_DISK;
no rows selected
SQL> alter diskgroup all mount;
alter diskgroup all mount
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15063: ASM discovered an insufficient number of disks for diskgroup “DATA”

Reason behind Error

Multipath storage devices were accessed by different paths on both nodes.

Solution : Add asm_diskstring = ‘/dev/oracleasm/disks/*’ at ASM init.ora file & bounce the box