Speeding-Up Oracle Export/Import Migration

Posted by Sagar Patil

Step 1 – export
Step 2 – import data
Step 3 – import everything else

1.Assuming a full export, use direct=y if you’re not using any predicates in the export.
Set your buffer to be big (10MB at least)
2.Alter all your constraints novalidate prior to export if you can bring the app down, and if you can take a consistent export. This helps in reinstating the constraints instantly without forcing Oracle to validate constraints on massive tables
3. set workarea_size_policy=manual
4. Set sort_area_size=6GB (yes, 6 GB). Combined with 4, it let Oracle build indexes with more space in memory for all the sorts without the need to spill to disk.
5. Set massive online redo logs at 2GB each, 2 members each, 6 groups. After the import, reset everything back down to “normal”.

 

You can locate amount of time remaining to finish an export using

select * from (
select opname, target, sofar, totalwork,
units, elapsed_seconds, totalwork/sofar – 1, round(elapsed_seconds * (totalwork/sofar -1)) Time_Remaining_Seconds, message
from v$session_longops order by start_time desc)
where rownum <=1;

Alternatively use TOAD and click on “Session Browser”

Look at Time Remaining to get a rough estimate of time.

RAC : Managing OCR Backup and Recovering OCR

Posted by Sagar Patil

Read more…

ASM Dynamic Views: ASM Instance Information

Posted by Sagar Patil

Read more…

RAC/CRS/Voting disk failover Tests

Posted by Sagar Patil

Read more…

How to recover from a Loss of Voting Disk

Posted by Sagar Patil


Loss of Voting Disk

Check where voting disks are located using “crsctl check crs”

Backup of voting disk : dd if=/dev/raw/votingdisk of=/vmasmtest/BACKUP/VOTING/votingdisk_06_may_07
dd: reading `/dev/raw/votingdisk’: No such device or address
305172+0 records in
305172+0 records out
[root@vmractest1 VOTING]# ls -l
total 152744
-rw-r–r– 1 oracle dba 156248064 May 6 16:40 votingdisk_06_may_07

Delete voting disks using rm command
Check RAC status “crs_stat -t”
Look into alrtlog messages at both instances and both Instance should show instance terminated.
Check available backups
Restore Voting Disk

Restore Voting Disk dd if=/vmasmtest/BACKUP/VOTING/votingdisk_06_may_07 of=/dev/raw/votingdisk
305172+0 records in
305172+0 records out

Restart CRS /etc/init.d/init.crs start

Check and Restart all Cluster components
./crsctl check crs
./crsctl query css votedisk
./crsctl start resources
Login into database & see everything is OK

Using ASM on command line

Posted by Sagar Patil

From the OS command prompt, type asmcmd, which brings up the ASM command-line prompt:

Read more…

Automatically start ASM/Database/EM/LSNR services

Posted by Sagar Patil
In theory there is no need to start services as they are brought online by Oracle Clusterware
Shell File dbora.txt

Copy dbora at /etc/init.d directory

# cd /etc/init.d # chmod 755 dbora # chown root:root dbora # ln -s /etc/init.d/dbora /etc/rc5.d/S99dbora # ln -s /etc/init.d/dbora /etc/rc0.d/K10dbora # ln -s /etc/init.d/dbora /etc/rc6.d/K10dbora

–  Change oratab entries from “N” to “Y” 

+ASM:/u01/app/oracle/product/10.1.0/db_1:Y      
TESTDB:/u01/app/oracle/product/10.1.0/db_1:Y   

–  Manually edit /etc/inittab so that entry to respawn init.cssd comes before running the runlevel 3

Orignal /etc/inittab file: (…)
# System initialization.
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6
(…)
h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null

Modified /etc/inittab file: (…)
# System initialization.
si::sysinit:/etc/rc.d/rc.sysinit
l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null
l3:3:wait:/etc/rc.d/rc 3
(…)

TAF Failover Configuration and Testing

Posted by Sagar Patil

Configure the service on RAC servers for a failover

TNS Client side config

PROD =
(DESCRIPTION =
(enable=broken)
(LOAD_BALANCE = yes)
(ADDRESS = (PROTOCOL = TCP)(HOST = oravip01.oracledbasupport.com)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = oravip02.oracledbasupport.com)(PORT = 1521))
(CONNECT_DATA =
(SERVICE_NAME = prod)
(failover_mode=(type=select)(method=basic))
)
)

Let’s test a Failover – Connect to an Oracle Instance 1 or 2

oracle@ora02 ~]$ showcrs
HA Resource Target State
———– —— —–
ora.ora01.ASM1.asm ONLINE ONLINE on ora01
ora.ora01.LISTENER_ora01.lsnr ONLINE ONLINE on ora01
ora.ora01.gsd ONLINE UNKNOWN on ora01
ora.ora01.ons ONLINE UNKNOWN on ora01
ora.ora01.vip ONLINE ONLINE on ora01
ora.ora02.ASM2.asm ONLINE ONLINE on ora02
ora.ora02.LISTENER_ora02.lsnr ONLINE ONLINE on ora02
ora.ora02.gsd ONLINE UNKNOWN on ora02
ora.ora02.ons ONLINE UNKNOWN on ora02
ora.ora02.vip ONLINE ONLINE on ora02
ora.prod.db ONLINE ONLINE on ora01
ora.prod.prod.cs ONLINE ONLINE on ora02
ora.prod.prod.prod1.srv ONLINE ONLINE on ora01
ora.prod.prod.prod2.srv ONLINE ONLINE on ora02
ora.prod.prod1.inst ONLINE ONLINE on ora01
ora.prod.prod2.inst ONLINE ONLINE on ora02

SQL> select instance_name from v$instance;
INSTANCE_NAME
—————-
prod2

[oracle@ora02 ~]$ crs_stop ora.prod.prod2.inst
Attempting to stop `ora.prod.prod2.inst` on member `ora02`
Stop of `ora.prod.prod2.inst` on member `ora02` succeeded.
At this stage the connections are diverted to prod1 instance.

SQL> select instance_name from v$instance;
INSTANCE_NAME
—————-
prod1

[oracle@ora02 ~]$ showcrs
HA Resource Target State
———– —— —–
ora.ora01.ASM1.asm ONLINE ONLINE on ora01
ora.ora01.LISTENER_ora01.lsnr ONLINE ONLINE on ora01
ora.ora01.gsd ONLINE UNKNOWN on ora01
ora.ora01.ons ONLINE UNKNOWN on ora01
ora.ora01.vip ONLINE ONLINE on ora01
ora.ora02.ASM2.asm ONLINE ONLINE on ora02
ora.ora02.LISTENER_ora02.lsnr ONLINE ONLINE on ora02
ora.ora02.gsd ONLINE UNKNOWN on ora02
ora.ora02.ons ONLINE UNKNOWN on ora02
ora.ora02.vip ONLINE ONLINE on ora01
ora.prod.db ONLINE ONLINE on ora01
ora.prod.prod.cs ONLINE ONLINE on ora02
ora.prod.prod.prod1.srv ONLINE ONLINE on ora01
ora.prod.prod.prod2.srv ONLINE OFFLINE
ora.prod.prod1.inst ONLINE ONLINE on ora01
ora.prod.prod2.inst OFFLINE OFFLINE

[oracle@ora02 ~]$ crs_start ora.prod.prod2.inst
Attempting to start `ora.prod.prod2.inst` on member `ora02`
Start of `ora.prod.prod2.inst` on member `ora02` succeeded.

What happens if Server is restarted?

I am connected to prod2 instance and a reboot migrates my connection to prod1 automatically.

SQL> select instance_name from v$instance;
INSTANCE_NAME
—————-
prod2

SQL> select count(*) from
(select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source)
COUNT(*)
———-
292465

SQL> select instance_name from v$instance;
INSTANCE_NAME
—————-
prod1

Let’s see how a RAC Load balancing works? Write a small sql test Script (verify.sql) like below

REM the following query is for TAF connection verification
col sid format 999
col serial# format 9999999
col failover_type format a13
col failover_method format a15
col failed_over format a11
SELECT   sid,
 serial#,
 failover_type,
 failover_method,
 failed_over
 FROM   v$session
 WHERE   username = 'SU';

REM the following query is for load balancing verification
SELECT   instance_name FROM v$instance;
exit

REM We can also combine two queries:
col inst_id format 999
col sid format 999
col serial# format 9999999
col failover_type format a13
col failover_method format a15
col failed_over format a11
SELECT   inst_id,
 sid,
 serial#,
 failover_type,
 failover_method,
 failed_over
 FROM   gv$session
 WHERE   username = 'SU';

REM a simple select to see the distribution of users when testing connection : load balancing
 SELECT   inst_id, COUNT ( * )
 FROM   gv$session
GROUP BY   inst_id;

Write loop.sh file to make number SQL connections. Please copy and paste at least 100 entries of line below. Oracle Listener will load balance connections by diverting new connections to least loaded oracle RAC instance.

nohup sqlplus system/0ra01@failover @verify.sql &
sleep 1
nohup sqlplus system/0ra01@failover @verify.sql &
sleep 1
nohup sqlplus system/0ra01@failover @verify.sql &
sleep 1
nohup sqlplus system/0ra01@failover @verify.sql &
sleep 1

Run loop.sh and note down connections shared between RAC 1 & RAC 2 nodes

[oracle@ora01 scripts]$ grep prod2 nohup.out | wc -l
35
[oracle@ora01 scripts]$ grep prod1 nohup.out | wc -l
41

 

RAC | How to use SRVCTL Command

Posted by Sagar Patil

Check out current configuration information

srvctl config database Displays the configuration information of the cluster database.
srvctl config service Displays the configuration information for the services.
srvctl config nodeapps Displays the configuration information for the node applications.
srvctl config asm Displays the configuration for the ASM instances on the node.

Summary of srvctl commands.

Command Targets Description
srvctl add
srvctl modify
srvctl remove
database
instance
service
nodeapps
srvctl add / remove adds/removes target‘s configuration information to/from the OCR.srvctl modify allows you to change some of target‘s configuration information in the OCR without wiping out the rest.
srvctl relocate service Allows you to reallocate a service from one named instance to another named instance.
srvctl config database
service
nodeapps
asm
Lists configuration information for target from the OCR.
srvctl disable
srvctl enable
database
instance
service
asm
srvctl disable disables target, meaning CRS will not consider it for automatic startup, failover, or restart. This option is useful to ensure an object that is down for maintenance is not accidentally automatically restarted.srvctl enable reenables the specified object.
srvctl getenv
srvctl setenv
srvctl unsetenv
database
instance
service
nodeapps
srvctl getenv displays the environment variables stored in the OCR for target.srvctl setenv allows these variables to be set, and unsetenv unsets them.
srvctl start
srvctl status
srvctl stop
database
instance
service
nodeapps
asm
Start, stop, or display status (started or stopped) of target.

Adding a Database Service
srvctl add service -d <database_name> -s <5ervice_name> -r “<preferred list>”

londonl$ srvctl add service -6 RAC -s SERVICE2 -i “RACl,RAC2” -a “RAC3,RAC4”

Starting a Database Service
srvctl start service -d <database_name> [-s “<service_name_li$t>” [-1 <in$t_name>]] [-0 <start_options>] [-c <connect_str> | -q]

londonl$ srvctl start service -d RAC -s “SERVICEl,SERVICE2”

Stopping a Database Service
srvctl stop service -d <database_name> [-s “<service_naine_list>” [-1 <inst_name>]] [-C <connect_str> | -q] [-f]

londonl$ srvctl stop service -d RAC -s “SERVICE2,SERVICE3” -f

Checking the Current Database Service Configuration
srvctl config service -d <database_name> [-s <service_name>] [-a] [-S <level>]

londonl$ srvctl config service -d RAC -a
The -a option includes information about the configuration of TAF for the database service

Checking Current Database Service Status
srvctl status service -d <name> -s “<service_name_list>” [-f] [-v] [-S <level>]

londonl$ srvctl status service -d RAC -s “SERVICEl,SERVICE4”

Enabling and Disabling a Database Service
srvctl disable service -d <database_name> -s “<service_name_list>” [-i <in$t_name>]

londonl$ srvctl disable service -d RAC -s SERVICE2 -i RAC4

srvctl enable service -d <database_name> -s “<service_name_list>” [-i <inst_name>]

londonl$ srvctl enable service -d RAC -s SERVICE2 -i RAC4

Removing a Database Service
srvctl remove service -d <database_name> -s <service_name> [ – i <inst_narne>] [-f]

londonl$ srvctl remove service -d RAC -s SERVICE4

Relocating a Database Service
srvctl relocate service -d <database_name> -s <service_name> -i <old_inst_name> -r <new_inst_name> [-f]

londonl$ srvctl relocate service -d RAC -s SERVICES -i RAC2 -t RAC4

Administering Instances

Starting an instance : srvctl start instance -d prod -i “prod1,prod2”

Stopping an instance: srvctl stop instance -d prod -i “prod1,prod2”

Checking the status of an instance : srvctl status instance -d prod – i “prod1,prod2”

Adding a new instance configuration : srvctl add instance -d prod – i prod3 -n prod3_node

Removing an existing instance configuration: srvctl remove instance -d prod3-i prod3_node

Disabling an instance: srvctl disable instance -d prod -i “prod1,prod2”

Enabling an instance : srvctl enable instance -d prod -i “prod1,prod2

Oracle Clusterware Log/Clusterware log files

Posted by Sagar Patil

In Oracle 10.2, Oracle Clusterware log files are created in the $ORA_CRS_HOME/log directory.

Read more…

Oracle Installer can’t discover ASM disks Ora-15186 Asmlib Error =[ASM_OPEN], ERROR = [1]

Posted by Sagar Patil

I could get ASM commandline to discover ASM disks at both RAC nodes. It can map VOL1, VOL2 at both nodes but Oracle RAC database installer won’t see ASM disks.

Operating System Linux Red Hat Advanced Server OS Version 4.0 Update Version Nahant Update 3
On WYGORA01
[root@wygora01 software]# /etc/init.d/oracleasm createdisk VOL1 /dev/mapper/mpath5
Marking disk “/dev/mapper/mpath5” as an ASM disk: [ OK ]
[root@wygora01 software]# /etc/init.d/oracleasm createdisk VOL1 /dev/mapper/mpath6
root@wygora01 software]# /etc/init.d/oracleasm createdisk VOL2 /dev/mapper/mpath6
Marking disk “/dev/mapper/mpath6” as an ASM disk: [ OK ]
On WYGORA02
[root@wygora02 ~]# /etc/init.d/oracleasm scandisks
Scanning system for ASM disks: [ OK ]

[root@wygora02 ~]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
[root@wygora01 ~]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
Initially I was trying to get oracle installer to create ASM instance. Somehow installer couldn’t list VOL1 & VOL2 disks though ASM listdisk could see it. After carrying ASM dropdisk/createdisk fixed it but restart was causing same issue. I decided to go ahead with installation and see if Oracle builds an environment . Installer managed to start ASM1 instance but failed to build ASM2 instance on another RAC node.

I tried starting the ASM2 instance manually and came across error

SQL> startup pfile=/u01/app/oracle/product/10.2.0/asm/admin/+ASM/pfile/init.ora
ASM instance started
Total System Global Area 130023424 bytes
Fixed Size 2019032 bytes
Variable Size 102838568 bytes
ASM Cache 25165824 bytes
ORA-15032: not all alterations performed
ORA-15063: ASM discovered an insufficient number of disks for diskgroup “DATA”
SQL> select PATH, MOUNT_STATUS, HEADER_STATUS, MODE_STATUS, STATE from V$ASM_DISK;
no rows selected
SQL> alter diskgroup all mount;
alter diskgroup all mount
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15063: ASM discovered an insufficient number of disks for diskgroup “DATA”

Reason behind Error

Multipath storage devices were accessed by different paths on both nodes.

Solution : Add asm_diskstring = ‘/dev/oracleasm/disks/*’ at ASM init.ora file & bounce the box

ORA-15186 ASMLIB ERROR FUNCTION = [ASM_OPEN], ERROR = [1], ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DATA"

Posted by Sagar Patil

Oracle Clusterware installed OK. ASM1 instance works OK while ASM2 would return ORA-15186/ORA-15063

Read more…

OCFS2 Support Guide Linux/Solaris

Posted by Sagar Patil

This support guide is a supplement to the OCFS2 User’s Guide and the OCFS2 FAQ. The information provided is directed towards support. End users should consult theUsers’ Guide and/or FAQ for information on setting up and using OCFS2.

Read more…

Creating Data Guard Primary/Standby on 2 Windows Servers

Posted by Sagar Patil

Oracle Data Guard is the management, monitoring, and automation software infrastructure that creates, maintains, and monitors one or more standby databases to protect enterprise data from failures, disasters, errors, and corruptions.

The process below is applicable if you have separate Windows/Unix  servers for Datguard primary & Standby Install . For single machine Dataguard Install use my other post.

Oracle Data Guard Overview

Windows 2000 with Oracle 9.2.0.8
Primary Server is called “Primary”
Standby Server is called “Standby”
A working DNS is available for both servers.
Oracle Instance is called DATAGUARD on both Servers
TNS Listener is configured as Primary.world and Standby.world

IMP:
– Configure listener.ora /Tnsnames.ora and add both Primary as well as Standby Database Services
– Always Copy the up -to- date password file from Primary to Standby
– Use Spfile for all configuration settings

Init file for Primary Database InitPrimary.ora
Init file for Standby Database Inittstandby.ora

Sample Listener file listener.ora
Sample Tnsnames file tnsnames.ora

At Primary Database

1. Check Archivelog mode and force database into Archivelog

SQLPLUS>archive log list;
If this returns “Noarchivelog Mode” then
Add following parameters at initPrimary.ora ….
log_archive_dest_1 = ‘location=f:\Oracle\oradata\Primary MANDATORY REOPEN=5’
log_archive_format = “Primary_%S.ARC”
log_archive_start = true
log_buffer = 1000000
log_checkpoint_interval = 10000
log_checkpoint_timeout = 1800

SQL>Startup nomount;
SQL>Alter database archivelog;
Do “alter system switch logfile” and see archive files are dunped at right location

SQL> archive log list;
Standbybase log mode Archive Mode
Automatic archival Enabled
Archive destination f:\Oracle\primary\DATAGUARD
Oldest online log sequence 14
Next log sequence to archive 17
Current log sequence 17

SQL> create spfile from pfile;
File created.

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

At this stage
– COLD backup Primary Database files and ship it to Standby Server.
– At primary create standby control file
SQLPLUS> Alter Database Create Standby Controlfile as ‘D:\Oracle\Oradata\Primary\control.ctl’;
– Copy standby.ctl on to Standby Machine

SQL> startup;
SQL> show parameter spfile;
NAME TYPE VALUE
———————————— ———– ——————————
spfile string %ORACLE_HOME%\StandbyBASE\SPFILE%
ORACLE_SID%.ORA

SQL> connect / as sysdba
Connected.

SQL> alter system set dg_broker_start=TRUE scope=both;
System altered.

C:\Documents and Settings\oracle>dgmgrl
DGMGRL for 32-bit Windows: Version 9.2.0.1.0 – Production.
(c) Copyright 2002 Oracle Corporation. All rights reserved.
Welcome to DGMGRL, type “help” for information.

DGMGRL> connect sys/oracle
Connected.

DGMGRL> show configuration verbose;
Error: ORA-16532: Standby Guard configuration does not exist

SQL> connect / as sysdba
Connected.

SQL> show parameter dg_broker_start
NAME TYPE VALUE
———————————— ———– ——————————
dg_broker_start boolean TRUE

DGMGRL> show configuration verbose;
Error: ORA-16532: Standby Guard configuration does not exist

Above Error indicates we need configuration at Data Guard

Configure Data Guard

Create a new DRC and define a primary Site

DGMGRL> create configuration ‘Primary Site’
as primary site is ‘Primary_site’
resource is ‘Primary_db’
hostname is ‘Primary’
instance name is ‘dataguard’
service name is ‘primary’
site is maintained as physical;
Configuration “Primary Site” added with primary site “Primary_site”
Standbybase resource “Primary_db” added.
Create Secondary Site

DGMGRL> create site ‘Standby_site’
resource is ‘Standby_db’
hostname is ‘Standby’
instance name is ‘dataguard’
service name is ‘standby’
site is maintained as physical;
Site “Standby_site” added to configuration.
Standbybase resource “Standby_db” added.
View DRC config

DGMGRL> show configuration verbose;
Configuration
Name: ‘Primary Site ‘
Enabled: ‘no’
Default state: ‘ONLINE’
Intended state: ‘OFFLINE’
Protection Mode: ‘MaxPerformance’
Number of sites: 2
Sites:
Primary Site: Primary_site
Standby Site: Standby_site
Current status for “Primary Site”:
DISABLED
Enable DRC
Please review alrtlog as well as log for DMON process($BDUMP/alrtDMON.log).

DGMGRL> enable configuration
Enabled.

DGMGRL> show configuration;
Configuration ‘Primary Site’ is
Primary Site is ‘Primary_site’
Standby Site is ‘Standby_site’
Current status for “Genesys Newport”:
SUCCESS

DGMGRL> show resource Primary_db
Resource ‘Primary_db’ on site ‘Primary_site’
depends on ‘Primary_site’
Current status for “Primary_db”:
SUCCESS

DGMGRL> show resource Standby_db
Resource ‘Standby_db’ on site ‘Standby_site’
depends on ‘Standby_site’
Current status for “Standby_db”:
SUCCESS

DGMGRL> exit

At Physical Standby Database

– Configure listener.ora /Tnsnames.ora and add both Primary as well as Standby Database Services
– Always Copy up -to- date password file from Primary to Standby
– Use Spfile for all Configuration Settings
=========================================================================

Add following 2 parameters at standby init.ora file
standby_file_management=auto
standby_archive_dest=’f:\oracle\oradata\standby’

C:\Documents and Settings\oracle>oradim -new -sid DATAGUARD -startmode m

C:\Documents and Settings\oracle>set oracle_sid=DATAGUARD

C:\Documents and Settings\oracle>sqlplus /nolog
SQL*Plus: Release 9.2.0.8.0 – Production on Wed Feb 14 16:12:33 2007
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.

SQL> connect / as sysdba
Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.
Total System Global Area 655843824 bytes
Fixed Size 455152 bytes
Variable Size 335544320 bytes
Database Buffers 318767104 bytes
Redo Buffers 1077248 bytes

SQL> alter database mount standby database;
Database altered.

SQL> recover managed standby database disconnect from session;
Media recovery complete.

SQL> archive log list;
Database log mode Archive Mode
Automatic archival Enabled
Archive destination f:\Oracle\oradata\primary
Oldest online log sequence 14
Next log sequence to archive 17
Current log sequence 17

SQL> create spfile from pfile;
File created.

SQL> shutdown immediate;
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.

SQL> startup mount;
ORACLE instance started.
Total System Global Area 655843824 bytes
Fixed Size 455152 bytes
Variable Size 335544320 bytes
Database Buffers 318767104 bytes
Redo Buffers 1077248 bytes
ORA-01666: controlfile is for a standby database
ORACLE instance shut down.

SQL> startup nomount;
ORACLE instance started.
Total System Global Area 655843824 bytes
Fixed Size 455152 bytes
Variable Size 335544320 bytes
Database Buffers 318767104 bytes
Redo Buffers 1077248 bytes

SQL> alter database mount standby database;
Database altered.

SQL> recover managed standby database disconnect from session;
Media recovery complete.

SQL> alter system set dg_broker_start=TRUE scope=both;
System altered.

SQL> show parameter dg_broker_start;
NAME TYPE VALUE
———————————— ———– ——————————
dg_broker_start boolean TRU

Init file for Primary Database InitPrimary.ora
Init file for Standby Database Inittstandby.ora

Sample Listener file listener.ora
Sample Tnsnames file tnsnames.ora

 

How to Install Statspack (8i/9i/10g)

Posted by Sagar Patil

Read more…

Locate Server Workload for a Defined Timeframe

Posted by Sagar Patil

Retrieve Stats for a Day from Statspack tables

select  to_char ( trunc ( b.snap_time )
 , 'DD-MM-YYYY' )
 , statistic#
 , name
 , sum ( value )
 from STATS$SYSSTAT A
 , stats$snapshot B
 where a.snap_id = b.snap_id
 and trunc ( b.snap_time ) > trunc ( sysdate - 30 )
 and a.statistic# = 54 < Replace
 with Stats
 Number Below > group
 by trunc ( b.snap_time )
 , statistic#
 , name
 order by trunc ( b.snap_time );

Please replace statistic from Type of stats

Select *
 from STATS$SYSSTAT
 where name like '%XXX%'

9 session logical reads
Physical Reads
54 Physical Reads
56 Physical reads direct
58 physical read bytes
39 physical read total bytes
42 physical write total bytes
66 physical write bytes
66 physical writes
CPU Related
355 OS Wait-cpu (latency) time
328 parse time cpu
8 recursive cpu usage

Rollback Related –
176 transaction tables consistent read rollbacks
180 rollbacks only – consistent read gets
181 cleanouts and rollbacks – consistent read gets
187 transaction rollbacks
5 user rollbacks
239 IMU CR rollbacks
186 rollback changes – undo records applied

How to find current Statspack level?

Posted by Sagar Patil

1. Look at table PERFSTST.STATS$SNAPSHOT
2. Run spreport.sql and you will notice it along with the snapids listed

Change Level
execute statspack.snap (i_snap_level=> 7, i_modify_parameter=>’true’);

Levels >= 0 General Performance Statistics
Any level greater than 0 collects general performance statistics, such as wait statistics, system events, system statistics, rollback segment data, row cache, SGA, background events, session events, lock statistics, buffer pool statistics, and parent latch statistics.

Levels >= 5 Additional Data: SQL Statements
This level includes all statistics gathered in the lower level(s), as well as performance data on SQL statements with high resource usage. In a level 5 snapshot (or higher), the time required for the snapshot to complete depends on the SHARED_POOL_SIZE and on the number of SQL statements in the shared pool at the time of the snapshot. The larger the shared pool, the longer it takes to complete the snapshot.

Levels >= 6 Additional Data: SQL Plans and SQL Plan Usage
This level includes all statistics gathered in the lower levels, as well a SQL plans and plan usage data for each of the high-resource SQL statements captured.
A level 6 snapshot gathers valuable information for determining whether the execution plan used for a SQL statement has changed. Therefore, level 6 snapshots should be used whenever a plan might have changed.

Levels >= 10 Additional Statistics: Parent and Child Latches
This level includes all statistics gathered in the lower levels, as well as parent and child latch information. Sometimes data gathered at this level can cause the snapshot to take longer to complete. This level can be resource-intensive, and it should only be used when advised by Oracle personnel.

Statspack threshold parameters:
* Number of executions of the SQL statement (default 100)
* Number of disk reads performed by the SQL statement (default 1,000)
* Number of parse calls performed by the SQL statement (default 1,000)
* Number of buffer gets performed by the SQL statement (default 10,000)
* Size of sharable memory used by the SQL statement (default 1 Mb)
* Version count for the SQL statement (default 20)

The SQL threshold levels used are either those stored in the table STATS$STATSPACK_PARAMETER or by the thresholds specified when the snapshot is taken.

How to define statspack level ?
SQL>  EXECUTE STATSPACK.SNAP(i_snap_level=>6);

Gather session statistics and wait events for a particular session
SQL>  EXECUTE STATSPACK.SNAP(i_session_id=>3);

Set new value as instance’s default
SQL>  EXECUTE STATSPACK.SNAP(i_snap_level=>10, i_modify_parameter=>’true’);

Switchover from primary to standby/ Switchover Primary : No Dgmgrl

Posted by Sagar Patil

This process will reverse database roles in a Data Guard setup, i.e. standby database becomes primary.

It’s assumed that:

  1. You already have working Data Guard setup between primary and standby
  2. Archive redo is begin shipped and applied from primary to standby
  3. Archive redo application is up to date between primary & standby

Requirements :

  1. All applications must be shutdown
  2. Switchover must be initiated on primary database before being completed on standby database
  3. The primary database must be open, the standby database must be mounted and in managed recovery mode

Note
When you perform a switchover the controlfile type is converted in-place, i.e. the primary controlfile becomes a standby controlfile and vice-versa. You must be CAREFUL which database you are working on…..

On the STANDBY database

1. Check archive_lag_target

SQL> show parameters lag

NAME TYPE VALUE ———————————— ———– —————————-
archive_lag_target integer 0

If non-zero set to zero using

SQL> alter system set archive_lag_target=0 scope=both;

On the PRIMARY database

1. Shutdown RAC databases

(RAC environment only) Shutdown all but one of the primary database instances.

2. Verify it is possible to switchover

SQL> SELECT SWITCHOVER_STATUS FROM V$DATABASE;

SWITCHOVER_STATUS

——————

SESSIONS ACTIVE

Switchover requires that there be only one active session (the one performing the switchover).

Documentation states the above query should return “TO STANDBY” for switchover to succeed but it – always returns “SESSIONS ACTIVE” because of the one active session – Problemo!!

The key point is to make sure there is only one user connected – SYS – check V$SESSION

3. Initiate the switchover

SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY WITH SESSION SHUTDOWN;

Database altered.

This converts the primary database to standby. Current controlfile is backed up to trace should you need to reverse the switchover. At this stage we temporarily have two standby databases…

4. Shutdown, restart and mount as standby

SQL> SHUTDOWN NORMAL;

ORA-01507: database not mounted <= This can be ignored

ORACLE instance shut down.

SQL> STARTUP NOMOUNT;
ORACLE instance started.
Total System Global Area 920912008 bytes
Fixed Size 711896 bytes
Variable Size 285212672 bytes
Database Buffers 111554492 bytes
Redo Buffers 811008 bytes

SQL> ALTER DATABASE MOUNT STANDBY DATABASE;

Database altered.

5. Verify switchover status

SQL> SELECT SWITCHOVER_STATUS FROM V$DATABASE;

SWITCHOVER_STATUS

——————

SESSIONS ACTIVE

According to documentation this should return “SWITCHOVER PENDING”, same catch as before, there has to be one session active to do the switchover.

6. Verify switchover status

SQL> SELECT SWITCHOVER_STATUS FROM V$DATABASE;

SWITCHOVER_STATUS

——————

SESSIONS ACTIVE

According to the documentation this should return “SWITCHOVER PENDING” but I have only ever seen “SESSIONS ACTIVE”, same situation as before – there has to be one session active to do the switchover.

On the STANDBY database

6. Switch standby database to primary

SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY ;

Database altered.

7. Shutdown and restart the new primary database

SQL> SHUTDOWN

ORA-01507: database not mounted <= This can be ignored

ORACLE instance shut down.

SQL> STARTUP
ORACLE instance started.
Total System Global Area 920912008 bytes
Fixed Size 711896 bytes
Variable Size 285212672 bytes
Database Buffers 111554492 bytes

Redo Buffers 811008 bytes
Database mounted.
Database opened.

The database is now the primary database.

On the STANDBY database (the original primary)

8. Start managed recovery and log apply services and set archive lag target

SQL> ALTER SYSTEM SET ARCHIVE_LAG_TARGET=900 SCOPE=BOTH;

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;

Database altered.

On the PRIMARY database (the original standby)

9. Begin archiving logs

SQL> ALTER SYSTEM ARCHIVE LOG START;

System altered.

SQL> ALTER SYSTEM SWITCH LOGFILE;

System altered.

On BOTH databases

10. Check the alert logs to make sure archived redo is being sent and received

The primary alert log should look like this

ARC1: Evaluating archive log 3 thread 1 sequence 30

ARC1: Archive destination LOG_ARCHIVE_DEST_2: Previously completed

ARC1: Beginning to archive log 3 thread 1 sequence 30

Creating archive destination LOG_ARCHIVE_DEST_1: ‘/u01/app/oracle/archive/arch/DATAGD_1_30.arc’

ARC1: Completed archiving log 3 thread 1 sequence 30

The standby alert log should look like this

Media Recovery Waiting for thread 1 seq# 30 (in transit)

Fri Sep 3 10:12:14 2004

Media Recovery Log  ‘/u01/app/oracle/archive/arch/DATAGD_1_30.arc

A database is creating more than 100 logs every single hour. What is happening ?

Posted by Sagar Patil

Solution: I tried looking into statspack report but it wouldn’t flag the insert/update/delete activity. Also statspack won’t record some of system activity which may contribute to redo logs.Only option to use was “LOGMINER”.

1. Start TOAD and click on DBA-> logminer

2. Select ftp directory else local files to mine

3. Enter FTP details of server

4. Look into file timestamp and select required files

5. I have selected nmst_0000000700.arc file

6. Click on “options” and Select options to display in TOAD window

7. Now click on Green arrow and logminer will start reading log files

8. Please be patient for some time and you would see a report

Oracle Performance Tuning | Effect of Optimizer_index_cost_adj parameter on Oracle Execution Plans

Posted by Sagar Patil

I have a SAP system with optimizer_index_cost_adj set to 10. Let’s look at Oracle execution plans and the resulting execution costs.

A value for “optimizer_index_cost_adj” =10 which will always favour index scans over full table scans. There are certain SQLs which will be better off with FULL Scan (with increased DB_FILE_MULTIBLOCK_READ_COUNT) over index scans.

select
a.average_wait c1,
b.average_wait c2,
a.total_waits /(a.total_waits + b.total_waits)*100 c3,
b.total_waits /(a.total_waits + b.total_waits)*100 c4
from
v$system_event a,
v$system_event b
where
a.event = ‘db file scattered read’
and
b.event = ‘db file sequential read’;

This system event indicate scattered reads (Full table Scans) are done on avg at 5.9 MilliSec while Index scans are done at 94 Milli secs.
We need to push oracle optimizer to evaluate what is better ? INDEX and FULL table scans but at a moment due to above parameter oracle will always go for index even though full scan could be quicker and less expensive. We are in a situation where we are almost using a RULE based optimizer.

I have spooled details on where oracle is spending it’s most time..
Scattered Reads – Most FULL table Scans which are not happening much
Sequential Reads – Index Scans which are very frequent

I have spooled avg value for OPTIMIZER_INDEX_COST_ADJ parameter using statspack system wait events.The calculations are purely based on time taken for “scattered and sequential reads”. I can say it’s value should be set to 86 looking at last 4 days statspack data.

Example on how FTS can be quicker than Index Scans

I have picked up a worst performer for analysis here. Following view “”VBAP_VAPMA”2 is based on VBAP and VAPMA tables, VBAP listed in top wait segments consistently. As we know Optimizer_index_cost_adj is favouring index scans even if they are worst performer over FULL table scan. I have done some calcualtions below.

SELECT “AEDAT”, “AUART”, “ERDAT”, “ERNAM”, “KONDM”, “KUNNR”, “MATKL”, “MATNR”,
“NETWR”, “POSNR”, “VBELN”, “VKORG”, “WAERK”, “ZZAD_LINE_STATUS”,
“ZZCDO”, “ZZCDO_P”, “ZZKONDM_P”
FROM SAPR3.”VBAP_VAPMA”
WHERE “MANDT” = :a0
AND “AEDAT” > :a1
AND “AUART” = :a2
AND “KONDM” = :a3
AND “VKORG” = :a4
AND “ZZCDO” >= :a5

Setting “Optimizer_index_cost_adj=100 changes execution plan from index “VBAP~Z3” to Full table sacn.

Optimizer_index_cost_adj=10 (Currently Set)
 SELECT STATEMENT Optimizer Mode=CHOOSE 2 313894
 TABLE ACCESS BY INDEX ROWID SAPR3.VAPMA 1 49 .4
 NESTED LOOPS 2 206 313893.8
 TABLE ACCESS BY INDEX ROWID SAPR3.VBAP 3 K 174 K 312568.2
 INDEX RANGE SCAN SAPR3.VBAP~Z3 15 M 100758
 INDEX RANGE SCAN SAPR3.VAPMA~Z01 1 3

Optimizer_index_cost_adj=100 (Oracle recommended Default Value)

SELECT STATEMENT Optimizer Mode=CHOOSE 2 577409
 TABLE ACCESS BY INDEX ROWID SAPR3.VAPMA 1 49 4
 NESTED LOOPS 2 206 577409
 TABLE ACCESS FULL SAPR3.VBAP 3 K 174 K 564153
 INDEX RANGE SCAN SAPR3.VAPMA~Z01 1 3

I will carry a simple calculations on how Oracle will estimate execution costs. Please note these are not precise formulas.

Approx Full Table Scan Cost : 484,193 Unadjusted
Cost here is calculated as “IO + CPU/1000 + NetIO*1.5” but a simple formula is (No of blocks/DB_FILE_MULTIBLOCK_READCOUNT)
(No of blocks/DB_FILE_MULTIBLOCK_READCOUNT)= 3,873,549 blocks/8 = 484,193

If we increase DB_FILE_MULTIBLOCK_READCOUNT to 32 + Reorg of table , cost of “FULL Scan” will drop to 82,000 giving 5 fold increase in IO.

Cost of an Index Scan : 149,483 is Adjusted value
It is using a non-unique index “SAPR3.VBAP~Z3” defined on columns MANDT, ZZBU_DIR, ZZBU_EDITION.
There are only 160 distinct values on this index out of 15.9 million rows –

“select MANDT, ZZBU_DIR, ZZBU_EDITION from SAPR3.vbap”

Index Range Scan Cost = blevel + (Avg leaf blk per key * (num_rows * selectivity))= 1,188,451 (Actual Value) > than FTS

Since we have set Optimizer_index_cost_adj=10, real cost we set is = 1,188,451*10/100= 118845.1 which is 10% of actual overhead

Final value of index cost must include efforts for accessing data blocks = Previous Cost + (Avg_data_blks_per_key * (Clustering_fact / Total Table blks))= 149,483

Conclusion:
We need to let oracle optimizer decide a best path for execution than forcing it to choose indexes all the time. Putting defualt value for “optimizer_index_cost_adj” must be followed with up-to-date stats as cost based optmizer is heavily dependent on right stats.

Top of Page

Top menu