11gR2 RAC-Dataguard Sync issue Between Primary & Standby

Posted by Sagar Patil

I have a 2 node RAC -DG setup between 2 remote data centres. After building DataGuard between them I am now coming across stange latency stats.

set pagesize 2000;
COLUMN NAME FORMAT A30;
COLUMN value FORMAT A20;
COLUMN UNIT FORMAT A20;
COLUMN time_computed FORMAT A20;
select name
 , value
 , unit
 , time_computed
 from v$dataguard_stats;

NAME                           VALUE                UNIT                 TIME_COMPUTED
------------------------------ -------------------- -------------------- --------------------
transport lag                  +00 00:02:21         day(2) to second(0)  06/15/2010 13:12:36
 interval
apply lag                      +00 00:02:23         day(2) to second(0)  06/15/2010 13:12:36
 interval
apply finish time              +00 00:00:01.846     day(2) to second(3)  06/15/2010 13:12:36
 interval
estimated startup time         16                   second               06/15/2010 13:12:36

The lag stats range from 2 mins to upto 3 hours.

select name
 , value
 , unit
 , time_computed
 from v$dataguard_stats;
 NAME                           VALUE                UNIT                 TIME_COMPUTED
------------------------------ -------------------- -------------------- --------------------
transport lag                  +00 00:53:10         day(2) to second(0)  06/15/2010 14:03:16
 interval
apply lag                      +00 00:53:12         day(2) to second(0)  06/15/2010 14:03:16
 interval
apply finish time              +00 00:00:01.846     day(2) to second(3)  06/15/2010 14:03:16
 interval
estimated startup time         16                   second               06/15/2010 14:03:16

DGMGRL> show database 'STDBY';
Database - STDBY
 Role:            PHYSICAL STANDBY
 Intended State:  APPLY-ON
 Transport Lag:   52 minutes 21 seconds
 Apply Lag:       52 minutes 23 seconds
 Real Time Query: OFF
 Instance(s):
 STDBY1 (apply instance)
 STDBY2
Database Status:
SUCCESS

DGMGRL> show database 'PROD' statusreport;
STATUS REPORT
 INSTANCE_NAME   SEVERITY ERROR_TEXT

DGMGRL> show database 'STDBY' statusreport;
STATUS REPORT
 INSTANCE_NAME   SEVERITY ERROR_TEXT

DGMGRL> show configuration verbose;
Configuration - dataguard
 Protection Mode: MaxPerformance
 Databases:
 PROD  - Primary database
 STDBY - Physical standby database
 Properties:
 FastStartFailoverThreshold      = '30'
 OperationTimeout                = '30'
 FastStartFailoverLagLimit       = '30'
 CommunicationTimeout            = '180'
 FastStartFailoverAutoReinstate  = 'TRUE'
 FastStartFailoverPmyShutdown    = 'TRUE'
 BystandersFollowRoleChange      = 'ALL'
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

DGMGRL> show database verbose 'PROD';
show database verbose 'STDBY';
Database - PROD
 Role:            PRIMARY
 Intended State:  TRANSPORT-ON
 Instance(s):
 PROD1
 PROD2
 Properties:
 DGConnectIdentifier             = 'PROD'
 ObserverConnectIdentifier       = ''
 LogXptMode                      = 'ASYNC'
 DelayMins                       = '0'
 Binding                         = 'optional'
 MaxFailure                      = '0'
 MaxConnections                  = '1'
 ReopenSecs                      = '300'
 NetTimeout                      = '30'
 RedoCompression                 = 'DISABLE'
 LogShipping                     = 'ON'
 PreferredApplyInstance          = ''
 ApplyInstanceTimeout            = '0'
 ApplyParallel                   = 'AUTO'
 StandbyFileManagement           = 'AUTO'
 ArchiveLagTarget                = '0'
 LogArchiveMaxProcesses          = '5'
 LogArchiveMinSucceedDest        = '1'
 DbFileNameConvert               = ''
 LogFileNameConvert              = ''
 FastStartFailoverTarget         = ''
 InconsistentProperties          = '(monitor)'
 InconsistentLogXptProps         = '(monitor)'
 SendQEntries                    = '(monitor)'
 LogXptStatus                    = '(monitor)'
 RecvQEntries                    = '(monitor)'
 SidName(*)
 StaticConnectIdentifier(*)
 StandbyArchiveLocation(*)
 AlternateLocation(*)
 LogArchiveTrace(*)
 LogArchiveFormat(*)
 TopWaitEvents(*)
 (*) - Please check specific instance for the property value
Database Status:
SUCCESS

DGMGRL>
Database - STDBY
 Role:            PHYSICAL STANDBY
 Intended State:  APPLY-ON
 Transport Lag:   28 minutes 40 seconds
 Apply Lag:       1 hour(s) 13 minutes 18 seconds
 Real Time Query: OFF
 Instance(s):
 STDBY1 (apply instance)
 STDBY2
 Properties:
 DGConnectIdentifier             = 'STDBY'
 ObserverConnectIdentifier       = ''
 LogXptMode                      = 'ARCH'
 DelayMins                       = '0'
 Binding                         = 'OPTIONAL'
 MaxFailure                      = '0'
 MaxConnections                  = '1'
 ReopenSecs                      = '300'
 NetTimeout                      = '30'
 RedoCompression                 = 'DISABLE'
 LogShipping                     = 'ON'
 PreferredApplyInstance          = ''
 ApplyInstanceTimeout            = '0'
 ApplyParallel                   = 'AUTO'
 StandbyFileManagement           = 'AUTO'
 ArchiveLagTarget                = '0'
 LogArchiveMaxProcesses          = '5'
 LogArchiveMinSucceedDest        = '1'
 DbFileNameConvert               = ''
 LogFileNameConvert              = ''
 FastStartFailoverTarget         = ''
 InconsistentProperties          = '(monitor)'
 InconsistentLogXptProps         = '(monitor)'
 SendQEntries                    = '(monitor)'
 LogXptStatus                    = '(monitor)'
 RecvQEntries                    = '(monitor)'
 SidName(*)
 StaticConnectIdentifier(*)
 StandbyArchiveLocation(*)
 AlternateLocation(*)
 LogArchiveTrace(*)
 LogArchiveFormat(*)
 TopWaitEvents(*)
 (*) - Please check specific instance for the property value
Database Status:
SUCCESS

DGMGRL> show database 'PROD' StatusReport
show database 'STDBY' StatusReport
STATUS REPORT
 INSTANCE_NAME   SEVERITY ERROR_TEXT

DGMGRL> STATUS REPORT
 INSTANCE_NAME   SEVERITY ERROR_TEXT

DGMGRL> show database 'PROD' InconsistentProperties
show database 'STDBY' InconsistentProperties
INCONSISTENT PROPERTIES
 INSTANCE_NAME        PROPERTY_NAME         MEMORY_VALUE         SPFILE_VALUE         BROKER_VALUE

DGMGRL>
INCONSISTENT PROPERTIES
 INSTANCE_NAME        PROPERTY_NAME         MEMORY_VALUE         SPFILE_VALUE         BROKER_VALUE

DGMGRL> DGMGRL> show database 'PROD' InconsistentLogXptProps
show database 'STDBY' InconsistentLogXptPropsINCONSISTENT LOG TRANSPORT PROPERTIES
 INSTANCE_NAME         STANDBY_NAME        PROPERTY_NAME         MEMORY_VALUE         BROKER_VALUE

DGMGRL> show database 'PROD' SendQEntries
PRIMARY_SEND_QUEUE
 STANDBY_NAME       STATUS     RESETLOGS_ID           THREAD              LOG_SEQ       TIME_GENERATED       TIME_COMPLETED    FIRST_CHANGE#     NEXT_CHANGE#       SIZE (KBs)
 STDBY     ARCHIVED        732278995                1                 4924  11/15/2010 09:29:54  11/15/2010 11:00:15        405196462        405290625            25968
 CURRENT        732278995                1                 4925  11/15/2010 11:00:15                             405290625                              6667
 CURRENT        732278995                2                 5016  11/15/2010 10:15:37                             405250777                             14538

DGMGRL> show database 'PROD' LogXptStatus
LOG TRANSPORT STATUS
PRIMARY_INSTANCE_NAME STANDBY_DATABASE_NAME               STATUS
 PROD1              STDBY
 PROD2              STDBY

DGMGRL> show database 'STDBY' RecvQEntries
STANDBY_RECEIVE_QUEUE
 STATUS     RESETLOGS_ID           THREAD              LOG_SEQ       TIME_GENERATED       TIME_COMPLETED    FIRST_CHANGE#     NEXT_CHANGE#       SIZE (KBs)
 PARTIALLY_APPLIED        732278995                1                 4924  11/15/2010 09:29:54  11/15/2010 11:00:15        405196462        405290625            25968

Conclusion:
The stats above show archives are sent over immediately so there is no issue with the transport or the apply on the Standby.
The archiving to the standby is done by the Archiver, so this is only when a logswitch occures. This is why there is a ‘lag’:
Using LGWR instead of ARCH will minimize the ‘Lag’ values

DataGuard Monitorable (Read-Only) Database Properties

Posted by Sagar Patil
Configuration details below are for a 2 Node RAC as Primary (PROD1,PROD2) & 2 node RAC(STDBY1,STDBY2) as Physical Standby.
DGMGRL> show database 'PROD';
Database - PROD
  Role:            PRIMARY
  Intended State:  TRANSPORT-ON
  Instance(s):
    PROD1
    PROD2
Database Status:
SUCCESS

DGMGRL> show database 'STDBY';
Database - STDBY
  Role:            PHYSICAL STANDBY
  Intended State:  APPLY-ON
  Transport Lag:   0 seconds
  Apply Lag:       6 seconds
  Real Time Query: OFF
  Instance(s):
    STDBY1 (apply instance)
    STDBY2
Database Status:
SUCCESS
DGMGRL> show instance verbose "PROD1";
Instance 'PROD1' of database 'PROD'
  Host Name: Primary-Server
  PFILE:
  Properties:
    SidName                         = 'PROD1'
    StaticConnectIdentifier         = '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=Primary-Server)(PORT=1529))(CONNECT_DATA=(SERVICE_NAME=PROD_DGMGRL)(INSTANCE_NAME=PROD1)(SERVER=DEDICATED)))'
    StandbyArchiveLocation          = '/mnt/logs/oradata/PROD/arch'
    AlternateLocation               = ''
    LogArchiveTrace                 = '0'
    LogArchiveFormat                = '%t_%s_%r.arc'
    TopWaitEvents                   = '(monitor)'
Instance Status:
SUCCESS

DGMGRL> show instance verbose "STDBY2";
Instance 'STDBY2' of database 'STDBY'
  Host Name : Standby-Server
  PFILE:
  Properties:
    SidName                         = 'STDBY2'
    StaticConnectIdentifier         = '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=Standby-Server)(PORT=1529))(CONNECT_DATA=(SERVICE_NAME=STDBY_DGMGRL)(INSTANCE_NAME=STDBY2)(SERVER=DEDICATED)))'
    StandbyArchiveLocation          = '/mnt/logs/oradata/PROD/arch'
    AlternateLocation               = ''
    LogArchiveTrace                 = '0'
    LogArchiveFormat                = '%t_%s_%r.arc'
    TopWaitEvents                   = '(monitor)'
Instance Status:
SUCCESS
Properties for Physical Standby Database :
1.  InconsistentLogXptProps (Inconsistent Redo Transport Properties)
2.  InconsistentProperties (Inconsistent Database Properties)
3.  LogXptStatus (Redo Transport Status)
4.  RecvQEntries (Receive Queue Entries)
5.  SendQEntries (Send Queue Entries)
6.  StatusReport (Status Report)
7.  TopWaitEvents

Properties for Logical Standby Database
1.  LsbyFailedTxnInfo (Logical Standby Failed Transaction Information)
2.  LsbyParameters (Logical Standby Parameters)
3.  LsbySkipTable (Logical Standby Skip Table)
4.  LsbySkipTxnTable (SQL Apply Skip Transaction Table)

1.  InconsistentLogXptProps (Inconsistent Redo Transport Properties)
    The InconsistentLogXptProps monitorable database property returns a table that shows all properties related to redo transport services whose values are inconsistent between the broker configuration file and the runtime value in the database.
	DGMGRL> show instance "PROD1" 'InconsistentLogXptProps';
2.  InconsistentProperties (Inconsistent Database Properties)
DGMGRL> show instance "PROD1" 'InconsistentProperties';
INCONSISTENT PROPERTIES
   INSTANCE_NAME        PROPERTY_NAME         MEMORY_VALUE         SPFILE_VALUE         BROKER_VALUE
DGMGRL> show instance "STDBY1" 'InconsistentProperties';
INCONSISTENT PROPERTIES
   INSTANCE_NAME        PROPERTY_NAME         MEMORY_VALUE         SPFILE_VALUE         BROKER_VALUE
3.  LogXptStatus (Redo Transport Status)
    The LogXptStatus monitorable database property returns a table that contains the error status of redo transport services for each of the enabled standby databases. This property pertains to the primary database.
	DGMGRL> show instance "PROD1" 'LogXptStatus';
		LOG TRANSPORT STATUS
		PRIMARY_INSTANCE_NAME STANDBY_DATABASE_NAME               STATUS
             PROD1              STDBY
             PROD2              STDBY
4.  RecvQEntries (Receive Queue Entries)
    The RecvQEntries monitorable database property returns a table indicating all log files that were received by the standby database but have not yet been applied.
    If no rows are returned, it implies all log files received have been applied. This property pertains to a standby database.
	DGMGRL> show instance "STDBY1" 'RecvQEntries';
STANDBY_RECEIVE_QUEUE
              STATUS     RESETLOGS_ID           THREAD              LOG_SEQ       TIME_GENERATED       TIME_COMPLETED    FIRST_CHANGE#     NEXT_CHANGE#       SIZE (KBs)
         NOT_APPLIED        732278995                2                 7733  09/13/2011 17:35:56  09/13/2011 17:36:07       7611896792       7611961172            39968
         NOT_APPLIED        732278995                2                 7734  09/13/2011 17:36:07  09/13/2011 17:36:16       7611961172       7612024484            39969
5. SendQEntries (Send Queue Entries)
   The SendQEntries monitorable database property returns a table that shows all log files on the primary database that were not successfully archived to one or more standby databases.
   This property pertains to the primary database.
	DGMGRL> show instance "PROD1" 'SendQEntries';
   PRIMARY_SEND_QUEUE
        STANDBY_NAME       STATUS     RESETLOGS_ID           THREAD              LOG_SEQ       TIME_GENERATED       TIME_COMPLETED    FIRST_CHANGE#     NEXT_CHANGE#       SIZE (KBs)
             STDBY     ARCHIVED        732278995                1                 7796  09/13/2011 17:37:31  09/13/2011 17:38:01       7612501278       7612699830              177
             STDBY     ARCHIVED        732278995                2                 7744  09/13/2011 17:37:46  09/13/2011 17:37:55       7612594908       7612657357            39970
             STDBY     ARCHIVED        732278995                2                 7745  09/13/2011 17:37:55  09/13/2011 17:38:05       7612657357       7612719596            39968
                       CURRENT         732278995                1                 7797  09/13/2011 17:38:01                            7612699830                                56
                       CURRENT         732278995                2                 7746  09/13/2011 17:38:05                            7612719596                             10491
6. StatusReport (Status Report)
   DGMGRL> SHOW DATABASE 'PROD1' 'StatusReport' ;
7. TopWaitEvents
   The TopWaitEvents monitorable database property specifies the 5 events with the longest waiting time in the specified instance. The events and their waiting time are retrieved from V$SYSTEM_EVENT.
DGMGRL> show instance "PROD1" 'TopWaitEvents';
TOP SYSTEM WAIT EVENTS
               Event            Wait Time
SQL*Net message from client   202752791770
   rdbms ipc message          65593825038
EMON slave idle wait          12015781443
  gcs remote message          5594728345
      DIAG idle wait          5590025706

DGMGRL> show instance "STDBY1" 'TopWaitEvents';
TOP SYSTEM WAIT EVENTS
               Event            		Wait Time
   rdbms ipc message          			16335113847
parallel recovery slave next change     5446704641
SQL*Net message from client           	4758412882
        PX Idle Wait           			2533146430
  gcs remote message           			1361762939

	

How to Safely Remove a Data Guard Broker Configuration under RAC/NON-RAC setup

Posted by Sagar Patil

1) Remove the Data Guard Broker Configuration

Using the Commandline DGMGRL
SQL> show parameter dg_broker;
NAME                                 TYPE        VALUE
———————————— ———– ——————————
dg_broker_config_file1               string      /mnt/data/oradata/PROD/dr1PROD.dat
dg_broker_config_file2               string      /mnt/data/oradata/PROD/dr2PROD.dat
dg_broker_start                      boolean     TRUE

DGMGRL> connect sys/sysgsadm
Connected.
DGMGRL> show configuration;
Configuration – dataguard
Protection Mode: MaxPerformance
Databases:
PROD  – Primary database
PROD – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

DGMGRL> remove configuration;
Removed configuration
DGMGRL> show configuration;
ORA-16532: Data Guard broker configuration does not exist
Configuration details cannot be determined by DGMGRL

2) On the primary database set dg_broker_start=false:

SQL> show parameter dg_broker

NAME                                 TYPE        VALUE
———————————— ———– ——————————
dg_broker_config_file1               string      ?/dbs/dr1@.dat
dg_broker_config_file2               string      ?/dbs/dr2@.dat
dg_broker_start                      boolean     TRUE

SQL> alter system set dg_broker_start=false scope=both;

System altered.

SQL> show parameter dg_broker;
NAME                                 TYPE        VALUE
———————————— ———– ——————————
dg_broker_config_file1               string      /mnt/data/oradata/PROD/dr1PROD.dat
dg_broker_config_file2               string      /mnt/data/oradata/PROD/dr2PROD.dat
dg_broker_start                      boolean     TRUE

3) On the primary disable any archive destinations that are archiving to the standby:

SQL> column A format 999;
column B format a40;
column C format a10;
column C word_wrapped;
column D word_wrapped;
select dest_id A,destination B,status C
from v$archive_dest
where target=’STANDBY’;
A B                                        C
—- —————————————- ———-
2 dg_PROD                                VALID

Primary> show parameter log_archive_dest_state_2;
NAME                                 TYPE        VALUE
———————————— ———– ——————————
log_archive_dest_state_2             string      ENABLE

Primary> alter system set log_archive_dest_state_2=defer scope=both;
System altered.

Primary> show parameter log_archive_dest_state_2;
NAME                                 TYPE        VALUE
———————————— ———– ——————————
log_archive_dest_state_2             string      DEFER

 

4) on the standby set the dg_broker_start parameter to false:

SQL> show parameter dg_broker;
NAME                                 TYPE        VALUE
———————————— ———– ——————————
dg_broker_config_file1               string      /mnt/data/oradata/PROD/dr1PROD.dat
dg_broker_config_file2               string      /mnt/data/oradata/PROD/dr2PROD.dat
dg_broker_start                      boolean     TRUE

SQL> alter system set dg_broker_start=false scope=both;
System altered.

5) On both system remove metadata files

oracle@hasunclu1:/u03/oracle/9.2.0/dbs> rm /mnt/data/oradata/PROD/dr1PROD.dat
oracle@hasunclu1:/u03/oracle/9.2.0/dbs> rm /mnt/data/oradata/PROD/dr2PROD.dat

If required you can activate standby database as below :

Standby> Shutdown abort;

Standby> startup;
ORACLE instance started.
Total System Global Area 9152860160 bytes
Fixed Size                  2234056 bytes
Variable Size            6610225464 bytes
Database Buffers         2516582400 bytes
Redo Buffers               23818240 bytes
Database mounted.
ORA-10456: cannot open standby database; media recovery session may be in progress

SQL> alter database activate standby database;
Database altered.

SQL> startup force;
ORACLE instance started.
Total System Global Area 9152860160 bytes
Fixed Size                  2234056 bytes
Variable Size            6610225464 bytes
Database Buffers         2516582400 bytes
Redo Buffers               23818240 bytes
Database mounted.
Database opened.

[oracle@Node2 PROD]$ srvctl stop database -d PROD
[oracle@Node2 PROD]$ srvctl start database -d PROD

[oracle@Node2 PROD]$ srvctl status instance -d PROD -n Node1
Instance PROD1 is running on node Node1
[oracle@Node2 PROD]$ srvctl status instance -d PROD -n Node2
Instance PROD2 is running on node Node2

The database won’t be opened by srvctl commands as it’s configured for STANDBY mode under Datauard setup.
Please change service mode using srvctl from MOUNT TO OPEN.

Script to Collect Data Guard Physical Standby Diagnostic Information [ID 241438.1]

Posted by Sagar Patil

I generally don’t cut and paste metalink notes. But in case you are using DG , for diagnosis you will have to use 2 metalink scripts, [ID 241438.1] & [ID 241374.1]

Script to Collect Data Guard Physical Standby Diagnostic Information [ID 241438.1]
 Modified 20-APR-2011     Type SCRIPT     Status PUBLISHED      
Overview
-------
This script is intended to provide an easy method to provide information
necessary to troubleshoot Data Guard issues.

-- NAME: DG_phy_stby_diag.sql    
-- ------------------------------------------------------------------------ 
-- AUTHOR:  
--    Michael Smith - Oracle Support Services - DataServer Group
--    Copyright 2002, Oracle Corporation      
-- ------------------------------------------------------------------------ 
-- PURPOSE: 
--    This script is to be used to assist in collection information to help
--    troubeshoot Data Guard issues.
-- ------------------------------------------------------------------------ 
-- DISCLAIMER: 
--    This script is provided for educational purposes only. It is NOT  
--    supported by Oracle World Wide Technical Support. 
--    The script has been tested and appears to work as intended. 
--    You should always run new scripts on a test instance initially. 
-- ------------------------------------------------------------------------ 
-- Script output is as follows:

set echo off
set feedback off
column timecol new_value timestamp
column spool_extension new_value suffix
select to_char(sysdate,'Mondd_hhmi') timecol,
'.out' spool_extension from sys.dual;
column output new_value dbname
select value || '_' output
from v$parameter where name = 'db_name';
spool dgdiag_phystby_&&dbname&&timestamp&&suffix
set lines 200
set pagesize 35
set trim on
set trims on
alter session set nls_date_format = 'MON-DD-YYYY HH24:MI:SS';
set feedback on
select to_char(sysdate) time from dual;

set echo on

--
-- ARCHIVER can be  (STOPPED | STARTED | FAILED) FAILED means that the archiver failed
-- to archive a -- log last time, but will try again within 5 minutes. LOG_SWITCH_WAIT
-- The ARCHIVE LOG/CLEAR LOG/CHECKPOINT event log switching is waiting for. Note that 
-- if ALTER SYSTEM SWITCH LOGFILE is hung, but there is room in the current online 
-- redo log, then value is NULL 

column host_name format a20 tru
column version format a9 tru
select instance_name,host_name,version,archiver,log_switch_wait from v$instance;

-- The following select will give us the generic information about how this standby is
-- setup.  The database_role should be standby as that is what this script is intended 
-- to be ran on.  If protection_level is different than protection_mode then for some
-- reason the mode listed in protection_mode experienced a need to downgrade.  Once the
-- error condition has been corrected the protection_level should match the protection_mode
-- after the next log switch.

column ROLE format a7 tru
select name,platform_id,database_role role,log_mode,
 flashback_on flashback,protection_mode,protection_level 
from v$database;

-- Force logging is not mandatory but is recommended.  Supplemental logging should be enabled
-- on the standby if a logical standby is in the configuration. During normal 
-- operations it is acceptable for SWITCHOVER_STATUS to be SESSIONS ACTIVE or NOT ALLOWED.

column force_logging format a13 tru
column remote_archive format a14 tru
column dataguard_broker format a16 tru
select force_logging,remote_archive,supplemental_log_data_pk,supplemental_log_data_ui,
switchover_status,dataguard_broker from v$database; 

-- This query produces a list of all archive destinations and shows if they are enabled,
-- what process is servicing that destination, if the destination is local or remote,
-- and if remote what the current mount ID is. For a physical standby we should have at
-- least one remote destination that points the primary set but it should be deferred.

COLUMN destination FORMAT A35 WRAP
column process format a7
column archiver format a8
column ID format 99

select dest_id "ID",destination,status,target,
archiver,schedule,process,mountid 
from v$archive_dest;

-- If the protection mode of the standby is set to anything higher than max performance
-- then we need to make sure the remote destination that points to the primary is set
-- with the correct options else we will have issues during switchover.

select dest_id,process,transmit_mode,async_blocks,
net_timeout,delay_mins,reopen_secs,register,binding
from v$archive_dest;

-- The following select will show any errors that occured the last time an attempt to
-- archive to the destination was attempted.  If ERROR is blank and status is VALID then
-- the archive completed correctly.

column error format a55 tru
select dest_id,status,error from v$archive_dest;

-- Determine if any error conditions have been reached by querying thev$dataguard_status
-- view (view only available in 9.2.0 and above):

column message format a80
select message, timestamp
from v$dataguard_status
where severity in ('Error','Fatal')
order by timestamp;

-- The following query is ran to get the status of the SRL's on the standby.  If the
-- primary is archiving with the LGWR process and SRL's are present (in the correct
-- number and size) then we should see a group# active.

select group#,sequence#,bytes,used,archived,status from v$standby_log;

-- The above SRL's should match in number and in size with the ORL's returned below:

select group#,thread#,sequence#,bytes,archived,status from v$log;

-- Query v$managed_standby to see the status of processes involved in the
-- configuration.

select process,status,client_process,sequence#,block#,active_agents,known_agents
from v$managed_standby;

-- Verify that the last sequence# received and the last sequence# applied to standby
-- database.

select al.thrd "Thread", almax "Last Seq Received", lhmax "Last Seq Applied"
from (select thread# thrd, max(sequence#) almax
 from v$archived_log
 where resetlogs_change#=(select resetlogs_change# from v$database)
 group by thread#) al,
 (select thread# thrd, max(sequence#) lhmax
 from v$log_history
 where first_time=(select max(first_time) from v$log_history)
 group by thread#) lh
where al.thrd = lh.thrd;

-- The V$ARCHIVE_GAP fixed view on a physical standby database only returns the next
-- gap that is currently blocking redo apply from continuing. After resolving the
-- identified gap and starting redo apply, query the V$ARCHIVE_GAP fixed view again
-- on the physical standby database to determine the next gap sequence, if there is
-- one.

select * from v$archive_gap;

-- Non-default init parameters.

set numwidth 5
column name format a30 tru
column value format a50 wra
select name, value
from v$parameter
where isdefault = 'FALSE';

spool off

Script to Collect Data Guard Primary Site Diagnostic Information [ID 241374.1]

Posted by Sagar Patil

I generally don’t cut and paste metalink notes. But in case you are using DG , for diagnosis you will have to use 2 metalink scripts, [ID 241438.1] & [ID 241374.1]

Script to Collect Data Guard Primary Site Diagnostic Information [ID 241374.1]
 Modified 20-APR-2011     Type SCRIPT     Status PUBLISHED      

Overview
--------
This script is intended to provide an easy method to provide information
necessary to troubleshoot Data Guard issues.

Script Notes
-------------

This script is intended to be run via sqlplus as the SYS or Internal user.

Script
-------
 - - - - - - - - - - - - - - - - Script begins here - - - - - - - - - - - - - - - -

-- NAME: dg_prim_diag.sql  (Run on PRIMARY with a LOGICAL or PHYSICAL STANDBY)
-- ------------------------------------------------------------------------ 
--    Copyright 2002, Oracle Corporation      
-- LAST UPDATED: 2/23/04
--
-- Usage: @dg_prim_diag
-- ------------------------------------------------------------------------ 
-- PURPOSE: 
--    This script is to be used to assist in collection information to help
--    troubeshoot Data Guard issues with an emphasis on Logical Standby.
-- ------------------------------------------------------------------------ 
-- DISCLAIMER: 
--    This script is provided for educational purposes only. It is NOT  
--    supported by Oracle World Wide Technical Support. 
--    The script has been tested and appears to work as intended. 
--    You should always run new scripts on a test instance initially. 
-- ------------------------------------------------------------------------ 
-- Script output is as follows:

set echo off
set feedback off
column timecol new_value timestamp
column spool_extension new_value suffix
select to_char(sysdate,'Mondd_hhmi') timecol,
'.out' spool_extension from sys.dual;
column output new_value dbname
select value || '_' output
from v$parameter where name = 'db_name';
spool dg_prim_diag_&&dbname&&timestamp&&suffix
set linesize 79
set pagesize 35
set trim on
set trims on
alter session set nls_date_format = 'MON-DD-YYYY HH24:MI:SS';
set feedback on
select to_char(sysdate) time from dual;

set echo on

-- In the following the database_role should be primary as that is what
-- this script is intended to be run on.  If protection_level is different
-- than protection_mode then for some reason the mode listed in
-- protection_mode experienced a need to downgrade.  Once the error
-- condition has been corrected the protection_level should match the
-- protection_mode after the next log switch.

column role format a7 tru
column name format a10 wrap

select name,platform_id,database_role role,log_mode,
 flashback_on flashback,protection_mode,protection_level 
from v$database;

-- ARCHIVER can be (STOPPED | STARTED | FAILED). FAILED means that the
-- archiver failed to archive a log last time, but will try again within 5
-- minutes. LOG_SWITCH_WAIT The ARCHIVE LOG/CLEAR LOG/CHECKPOINT event log
-- switching is waiting for.  Note that if ALTER SYSTEM SWITCH LOGFILE is
-- hung, but there is room in the current online redo log, then value is
-- NULL

column host_name format a20 tru
column version format a9 tru

select instance_name,host_name,version,archiver,log_switch_wait
from v$instance;

-- The following query give us information about catpatch.
-- This way we can tell if the procedure doesn't match the image.

select version, modified, status from dba_registry
where comp_id = 'CATPROC';

-- Force logging is not mandatory but is recommended.  Supplemental
-- logging must be enabled if the standby associated with this primary is
-- a logical standby. During normal operations it is acceptable for
-- SWITCHOVER_STATUS to be SESSIONS ACTIVE or TO STANDBY.

column force_logging format a13 tru
column remote_archive format a14 tru
column dataguard_broker format a16 tru

select force_logging,remote_archive,
 supplemental_log_data_pk,supplemental_log_data_ui,
 switchover_status,dataguard_broker
from v$database; 

-- This query produces a list of all archive destinations.  It shows if
-- they are enabled, what process is servicing that destination, if the
-- destination is local or remote, and if remote what the current mount ID
-- is.

column destination format a35 wrap
column process format a7
column archiver format a8
column ID format 99
column mid format 99

select dest_id "ID",destination,status,target,
 schedule,process,mountid  mid
from v$archive_dest order by dest_id;

-- This select will give further detail on the destinations as to what
-- options have been set.  Register indicates whether or not the archived
-- redo log is registered in the remote destination control file.

set numwidth 8
column ID format 99

select dest_id "ID",archiver,transmit_mode,affirm,async_blocks async,
 net_timeout net_time,delay_mins delay,reopen_secs reopen,
 register,binding
from v$archive_dest order by dest_id;

-- The following select will show any errors that occured the last time
-- an attempt to archive to the destination was attempted.  If ERROR is
-- blank and status is VALID then the archive completed correctly.

column error format a55 wrap

select dest_id,status,error from v$archive_dest;

-- The query below will determine if any error conditions have been
-- reached by querying the v$dataguard_status view (view only available in
-- 9.2.0 and above):

column message format a80

select message, timestamp
from v$dataguard_status
where severity in ('Error','Fatal')
order by timestamp;

-- The following query will determine the current sequence number
-- and the last sequence archived.  If you are remotely archiving
-- using the LGWR process then the archived sequence should be one
-- higher than the current sequence.  If remotely archiving using the
-- ARCH process then the archived sequence should be equal to the
-- current sequence.  The applied sequence information is updated at
-- log switch time.

select ads.dest_id,max(sequence#) "Current Sequence",
 max(log_sequence) "Last Archived"
 from v$archived_log al, v$archive_dest ad, v$archive_dest_status ads
 where ad.dest_id=al.dest_id
 and al.dest_id=ads.dest_id
 and al.resetlogs_change#=(select max(resetlogs_change#) from v$archived_log )
 group by ads.dest_id;

-- The following select will attempt to gather as much information as
-- possible from the standby.  SRLs are not supported with Logical Standby
-- until Version 10.1.

set numwidth 8
column ID format 99
column "SRLs" format 99
column Active format 99

select dest_id id,database_mode db_mode,recovery_mode,
 protection_mode,standby_logfile_count "SRLs",
 standby_logfile_active ACTIVE,
 archived_seq#
from v$archive_dest_status;

-- Query v$managed_standby to see the status of processes involved in
-- the shipping redo on this system.  Does not include processes needed to
-- apply redo.

select process,status,client_process,sequence#
from v$managed_standby;

-- The following query is run on the primary to see if SRL's have been
-- created in preparation for switchover.

select group#,sequence#,bytes from v$standby_log;

-- The above SRL's should match in number and in size with the ORL's
-- returned below:

select group#,thread#,sequence#,bytes,archived,status from v$log;

-- Non-default init parameters.

set numwidth 5
column name format a30 tru
column value format a48 wra
select name, value
from v$parameter
where isdefault = 'FALSE';

spool off

 - - - - - - - - - - - - - - - -  Script ends here  - - - - - - - - - - - - - - - -

 

 

11g RAC | Using Duplicate target database 11g Active Database option

Posted by Sagar Patil

I have a 2 Node RAC Standby database (STDBY) . I need to replicate it as a Load TEST database (LDTEST) in a read/write mode.

I will run thru following steps:
1. Preparing the Auxiliary Instance
2. Starting and Configuring RMAN Before Duplication
3. Duplicating a Database

1. Preparing the Auxiliary Instance

Step 1: Create an Oracle Password File for the Auxiliary Instance

When using FROM ACTIVE DATABASE option the source database instance which is the database instance to which RMAN is connected as TARGET connects directly to the auxiliary database instance.  This connection requires a password file with the same SYSDBA password.

[oracle@Node3]$ pwd
/mnt/data/oradata/LDTEST   — The password file placed at Clustered storage.
[oracle@Node3]$ cp orapwSTDBY ../LDTEST
[oracle@Node3]$ cd ../LDTEST
[oracle@Node3]$ ls -lrt
total 4
-rw-r—– 1 oracle oinstall 1536 May 3 12:57 orapwSTDBY

[oracle@Node3]$ mv orapwSTDBY orapwLDTTEST
[oracle@Node3]$ ls -lrt
total 4
-rw-r—– 1 oracle oinstall 1536 May 3 12:57 orapwLDTTEST

Step 2: Establish Oracle Net Connectivity to the Auxiliary Instance

When duplicating from an active database, you must first have connected as SYSDBA to the auxiliary instance by means of a net service name.
Add new database instance details $ORACLE_HOME/netaork/admin/listener.ora

(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = LDTEST) # Replicated DB
(ORACLE_HOME = /opt/app/oracle/product/11.2/db_1)
(SID_NAME =LDTTEST1)
)

[oracle@Node3 admin]$ lsnrctl reload
[oracle@Node3 admin]$ lsnrctl status
Service “LDTEST” has 1 instance(s).
Instance “LDTTEST1”, status UNKNOWN, has 1 handler(s) for this service…
Service “STDBY_DGMGRL” has 1 instance(s).
Instance “STDBY1”, status UNKNOWN, has 1 handler(s) for this service…
The command completed successfully

Add following dedicated entry at /opt/app/oracle/product/11.2/db_1/network/admin

LDTTEST1=
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = Node3scan)(PORT = 1529))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = LDTEST)
)
)

[oracle@Node3 admin]$ tnsping LDTTEST1
TNS Ping Utility for Linux: Version 11.2.0.2.0 – Production on 03-MAY-2011
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = Node3scan)(PORT = 1529)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = LDTEST)))
OK (10 msec)

Step 3: Create an Initialization Parameter File for An Auxiliary Instance

Change directories at pfile to point to new database directory structure
** : set *.cluster_database=false
Let’s create dump directories needed. The easiest way is to copy structure of directory tree from existing instance.

[oracle@Node3 STDBY]$ pwd
/opt/app/oracle/diag/rdbms/STDBY
find . -type d -exec mkdir /opt/app/oracle/diag/rdbms/LDTEST/{} \;
“du -a” showed right directory structure created
84 ./LDTTEST1/trace
4 ./LDTTEST1/sweep
4 ./LDTTEST1/metadata
4 ./LDTTEST1/alert
4 ./LDTTEST1/stage
4 ./LDTTEST1/hm
4 ./LDTTEST1/incident
136 ./LDTTEST1

SQL> create pfile=’$ORACLE_HOME/dbs/initLDTTEST1.ora’ from spfile;

Edit pfile and make directory location changes required for new Database.

Step 4: Start Auxiliary Instance with SQL*Plus

SQL> startup nomount;
ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance
ORACLE instance started.
Total System Global Area 9152860160 bytes
Fixed Size 2234056 bytes
Variable Size 6945769784 bytes
Database Buffers 2181038080 bytes
Redo Buffers 23818240 bytes
SQL> show parameter db_name
NAME TYPE VALUE
———————————— ———– ——————————
db_name string LDTTEST1

2. Starting and Configuring RMAN Before Duplication
Step 1: Start RMAN and Connect to the Database Instances
Step 2: Mount or Open the Source Database
Step 1: Start RMAN and Connect to the Database Instances

RMAN> connect target sys/sysgsadm@STDBY
connected to target database: PROD (DBID=4020163110)
RMAN> CONNECT AUXILIARY SYS/sysgsadm@LDTTEST1
connected to auxiliary database: LDTTEST1 (not mounted)

Step 2: Mount or Open the Source Database

Before beginning RMAN duplication, mount or open the source database it if it is not already mounted or open.

3. Duplicating a Database

Run following RMAN command

RMAN> DUPLICATE TARGET DATABASE TO LDTEST
FROM ACTIVE DATABASE
DB_FILE_NAME_CONVERT ‘/PROD’,’/LDTEST’
PFILE=’/opt/app/oracle/product/11.2/db_1/dbs/initLDTTEST1.ora’;

Starting Duplicate Db at 03-MAY-11
using target database control file instead of recovery catalog
allocated channel: ORA_AUX_DISK_1
channel ORA_AUX_DISK_1: SID=392 device type=DISK
allocated channel: ORA_AUX_DISK_2
channel ORA_AUX_DISK_2: SID=490 device type=DISK
allocated channel: ORA_AUX_DISK_3
channel ORA_AUX_DISK_3: SID=586 device type=DISK
contents of Memory Script:
{
sql clone “create spfile from memory”;
}
executing Memory Script
sql statement: create spfile from memory
contents of Memory Script:
{
shutdown clone immediate;
startup clone nomount;
}
executing Memory Script
Oracle instance shut down
connected to auxiliary database (not started)
Oracle instance started
Total System Global Area 9152860160 bytes
Fixed Size 2234056 bytes
Variable Size 6979324216 bytes
Database Buffers 2147483648 bytes
Redo Buffers 23818240 bytes
contents of Memory Script:
{
sql clone “alter system set db_name =
”PROD” comment=
”Modified by RMAN duplicate” scope=spfile”;
sql clone “alter system set db_unique_name =
”LDTEST” comment=
”Modified by RMAN duplicate” scope=spfile”;
shutdown clone immediate;
startup clone force nomount
backup as copy current controlfile auxiliary format ‘/mnt/data/oradata/LDTEST/control01.ctl’;
restore clone controlfile to ‘/mnt/data/oradata/LDTEST/control02.ctl’ from
‘/mnt/data/oradata/LDTEST/control01.ctl’;
alter clone database mount;
}
executing Memory Script
sql statement: alter system set db_name = ”PROD” comment= ”Modified by RMAN duplicate” scope=spfile
sql statement: alter system set db_unique_name = ”LDTEST” comment= ”Modified by RMAN duplicate” scope=spfile
Oracle instance shut down
Oracle instance started
Total System Global Area 9152860160 bytes
Fixed Size 2234056 bytes
Variable Size 6979324216 bytes
Database Buffers 2147483648 bytes
Redo Buffers 23818240 bytes
Starting backup at 03-MAY-11
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=300 instance=STDBY1 device type=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: SID=396 instance=STDBY1 device type=DISK
allocated channel: ORA_DISK_3
channel ORA_DISK_3: SID=495 instance=STDBY1 device type=DISK
channel ORA_DISK_1: starting datafile copy
copying current control file
output file name=/opt/app/oracle/product/11.2/db_1/dbs/snapcf_STDBY1.f tag=TAG20110503T162228 RECID=13 STAMP=750183751
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:07
Finished backup at 03-MAY-11
Starting restore at 03-MAY-11
allocated channel: ORA_AUX_DISK_1
channel ORA_AUX_DISK_1: SID=392 device type=DISK
allocated channel: ORA_AUX_DISK_2
channel ORA_AUX_DISK_2: SID=490 device type=DISK
allocated channel: ORA_AUX_DISK_3
channel ORA_AUX_DISK_3: SID=586 device type=DISK
channel ORA_AUX_DISK_2: skipped, AUTOBACKUP already found
channel ORA_AUX_DISK_3: skipped, AUTOBACKUP already found
channel ORA_AUX_DISK_1: copied control file copy
Finished restore at 03-MAY-11
database mounted
contents of Memory Script:
{
set newname for datafile 1 to
“/mnt/data/oradata/LDTEST/system01.dbf”;
set newname for datafile 2 to
“/mnt/data/oradata/LDTEST/sysaux01.dbf”;
set newname for datafile 3 to
“/mnt/data/oradata/LDTEST/undotbs01.dbf”;
set newname for datafile 4 to
“/mnt/data/oradata/LDTEST/users01.dbf”;
set newname for datafile 5 to
“/mnt/data/oradata/LDTEST/undotbs02.dbf”;
backup as copy reuse
datafile 1 auxiliary format
“/mnt/data/oradata/LDTEST/system01.dbf” datafile
2 auxiliary format
“/mnt/data/oradata/LDTEST/sysaux01.dbf” datafile
3 auxiliary format
“/mnt/data/oradata/LDTEST/undotbs01.dbf” datafile
4 auxiliary format
“/mnt/data/oradata/LDTEST/users01.dbf” datafile
5 auxiliary format
“/mnt/data/oradata/LDTEST/undotbs02.dbf” datafile
}
executing Memory Script
Starting backup at 03-MAY-11
using channel ORA_DISK_1
using channel ORA_DISK_2
using channel ORA_DISK_3
channel ORA_DISK_1: starting datafile copy
input datafile file number=00002 name=/mnt/data/oradata/PROD/sysaux01.dbf
channel ORA_DISK_2: starting datafile copy
input datafile file number=00001 name=/mnt/data/oradata/PROD/system01.dbf
output file name=/mnt/data/oradata/LDTEST/system01.dbf tag=TAG20110503T162300
channel ORA_DISK_3: datafile copy complete, elapsed time: 00:04:05
channel ORA_DISK_3: starting datafile copy
input datafile file number=00005 name=/mnt/data/oradata/PROD/undotbs02.dbf
output file name=/mnt/data/oradata/LDTEST/sysaux01.dbf tag=TAG20110503T162300
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:06:50
channel ORA_DISK_1: starting datafile copy
input datafile file number=00003 name=/mnt/data/oradata/PROD/undotbs01.dbf
output file name=/mnt/data/oradata/LDTEST/cdc_data01.dbf tag=TAG20110503T162300
channel ORA_DISK_2: datafile copy complete, elapsed time: 00:02:15
channel ORA_DISK_2: starting datafile copy
input datafile file number=00004 name=/mnt/data/oradata/PROD/users01.dbf
output file name=/mnt/data/oradata/LDTEST/undotbs01.dbf tag=TAG20110503T162300
Finished backup at 03-MAY-11
sql statement: alter system archive log current
contents of Memory Script:
{
backup as copy reuse
archivelog like “/mnt/logs/oradata/PROD/arch/2_753_747681489.arc” auxiliary format
“/mnt/logs/oradata/LDTEST/arch/2_753_747681489.arc” archivelog like
“/mnt/logs/oradata/PROD/arch/1_664_747681489.arc” auxiliary format
“/mnt/logs/oradata/LDTEST/arch/1_664_747681489.arc” archivelog like
“/mnt/logs/oradata/PROD/arch/2_754_747681489.arc” auxiliary format
“/mnt/logs/oradata/LDTEST/arch/2_754_747681489.arc” ;
catalog clone archivelog “/mnt/logs/oradata/LDTEST/arch/2_753_747681489.arc”;
catalog clone archivelog “/mnt/logs/oradata/LDTEST/arch/1_664_747681489.arc”;
catalog clone archivelog “/mnt/logs/oradata/LDTEST/arch/2_754_747681489.arc”;
switch clone datafile all;
}
executing Memory Script
Starting backup at 03-MAY-11
using channel ORA_DISK_1
using channel ORA_DISK_2
using channel ORA_DISK_3
channel ORA_DISK_1: starting archived log copy
input archived log thread=2 sequence=753 RECID=782 STAMP=750183649
channel ORA_DISK_2: starting archived log copy
input archived log thread=1 sequence=664 RECID=784 STAMP=750184270
channel ORA_DISK_3: starting archived log copy
input archived log thread=2 sequence=754 RECID=786 STAMP=750184271
output file name=/mnt/logs/oradata/LDTEST/arch/2_753_747681489.arc RECID=0 STAMP=0
channel ORA_DISK_1: archived log copy complete, elapsed time: 00:00:02
output file name=/mnt/logs/oradata/LDTEST/arch/1_664_747681489.arc RECID=0 STAMP=0
channel ORA_DISK_2: archived log copy complete, elapsed time: 00:00:01
output file name=/mnt/logs/oradata/LDTEST/arch/2_754_747681489.arc RECID=0 STAMP=0
channel ORA_DISK_3: archived log copy complete, elapsed time: 00:00:01
Finished backup at 03-MAY-11
cataloged archived log
archived log file name=/mnt/logs/oradata/LDTEST/arch/2_753_747681489.arc RECID=783 STAMP=750184304
cataloged archived log
archived log file name=/mnt/logs/oradata/LDTEST/arch/1_664_747681489.arc RECID=784 STAMP=750184304
cataloged archived log
archived log file name=/mnt/logs/oradata/LDTEST/arch/2_754_747681489.arc RECID=785 STAMP=750184305
datafile 1 switched to datafile copy
input datafile copy RECID=13 STAMP=750184307 file name=/mnt/data/oradata/LDTEST/system01.dbf
datafile 2 switched to datafile copy
input datafile copy RECID=14 STAMP=750184308 file name=/mnt/data/oradata/LDTEST/sysaux01.dbf
datafile 3 switched to datafile copy
input datafile copy RECID=15 STAMP=750184309 file name=/mnt/data/oradata/LDTEST/undotbs01.dbf
datafile 4 switched to datafile copy
input datafile copy RECID=16 STAMP=750184309 file name=/mnt/data/oradata/LDTEST/users01.dbf
datafile 5 switched to datafile copy
input datafile copy RECID=17 STAMP=750184310 file name=/mnt/data/oradata/LDTEST/undotbs02.dbf
contents of Memory Script:
{
set until scn 263980944;
recover
clone database
delete archivelog
;
}
executing Memory Script
executing command: SET until clause
Starting recover at 03-MAY-11
using channel ORA_AUX_DISK_1
using channel ORA_AUX_DISK_2
using channel ORA_AUX_DISK_3
starting media recovery
archived log for thread 1 with sequence 664 is already on disk as file /mnt/logs/oradata/LDTEST/arch/1_664_747681489.arc
archived log for thread 2 with sequence 754 is already on disk as file /mnt/logs/oradata/LDTEST/arch/2_754_747681489.arc
archived log file name=/mnt/logs/oradata/LDTEST/arch/1_664_747681489.arc thread=1 sequence=664
archived log file name=/mnt/logs/oradata/LDTEST/arch/2_754_747681489.arc thread=2 sequence=754
media recovery complete, elapsed time: 00:00:03
Finished recover at 03-MAY-11
Oracle instance started
Total System Global Area 9152860160 bytes
Fixed Size 2234056 bytes
Variable Size 6945769784 bytes
Database Buffers 2181038080 bytes
Redo Buffers 23818240 bytes
sql statement: CREATE CONTROLFILE REUSE SET DATABASE “LDTEST” RESETLOGS ARCHIVELOG
MAXLOGFILES 192
MAXLOGMEMBERS 3
MAXDATAFILES 2048
MAXINSTANCES 32
MAXLOGHISTORY 1168
LOGFILE
GROUP 1 SIZE 50 M ,
GROUP 2 SIZE 50 M
DATAFILE
‘/mnt/data/oradata/LDTEST/system01.dbf’
CHARACTER SET AL32UTF8
sql statement: ALTER DATABASE ADD LOGFILE
INSTANCE ‘i2’
GROUP 3 SIZE 50 M ,
GROUP 4 SIZE 50 M
contents of Memory Script:
{
set newname for tempfile 1 to
“/mnt/data/oradata/LDTEST/temp01.dbf”;
switch clone tempfile all;
catalog clone datafilecopy “/mnt/data/oradata/LDTEST/sysaux01.dbf”,
“/mnt/data/oradata/LDTEST/undotbs01.dbf”,
“/mnt/data/oradata/LDTEST/users01.dbf”,
“/mnt/data/oradata/LDTEST/undotbs02.dbf”,
switch clone datafile all;
}
executing Memory Script
executing command: SET NEWNAME
renamed tempfile 1 to /mnt/data/oradata/LDTEST/temp01.dbf in control file
cataloged datafile copy
datafile copy file name=/mnt/data/oradata/LDTEST/sysaux01.dbf RECID=1 STAMP=750184356
cataloged datafile copy
datafile copy file name=/mnt/data/oradata/LDTEST/undotbs01.dbf RECID=2 STAMP=750184356
cataloged datafile copy
datafile copy file name=/mnt/data/oradata/LDTEST/users01.dbf RECID=3 STAMP=750184356
cataloged datafile copy
datafile copy file name=/mnt/data/oradata/LDTEST/undotbs02.dbf RECID=4 STAMP=750184357
cataloged datafile copy
datafile 2 switched to datafile copy
input datafile copy RECID=1 STAMP=750184356 file name=/mnt/data/oradata/LDTEST/sysaux01.dbf
datafile 3 switched to datafile copy
input datafile copy RECID=2 STAMP=750184356 file name=/mnt/data/oradata/LDTEST/undotbs01.dbf
datafile 4 switched to datafile copy
input datafile copy RECID=3 STAMP=750184356 file name=/mnt/data/oradata/LDTEST/users01.dbf
datafile 5 switched to datafile copy
input datafile copy RECID=4 STAMP=750184357 file name=/mnt/data/oradata/LDTEST/undotbs02.dbf
Reenabling controlfile options for auxiliary database
Executing: alter database add supplemental log data(PRIMARY KEY, UNIQUE) columns
Executing: alter database force logging
contents of Memory Script:
{
Alter clone database open resetlogs;
}
executing Memory Script
database opened
Finished Duplicate Db at 03-MAY-11

The database is now working fine on One Node1, I will have to convert it into a 2 node RAC database.

Create shared spfile for both instances , set CLUSTER_DATABASE to TRUE at spfile/pfile

SQL> alter system set cluster_database=TRUE scope=spfile;
System altered.
SQL> shutdown abort;
ORACLE instance shut down.
[oracle@Node3 dbs]$ cat initLDTTEST1.ora
SPFILE=’/mnt/data/oradata/LDTEST/spfileLDTTEST.ora'[oracle@Node4 dbs]$ cat initLDTTEST2.ora
SPFILE=’/mnt/data/oradata/LDTEST/spfileLDTTEST.ora’

Move password file to clustered shared storage and create soft links from both nodes Node3, Node4 to orapwLDTTEST

[oracle@Node3 dbs]$ ln -s /mnt/data/oradata/LDTEST/orapwLDTTEST orapwLDTTEST2
[oracle@Node3 dbs]$ ln -s /mnt/data/oradata/LDTEST/orapwLDTTEST orapwLDTTEST1
[oracle@Node3 dbs]$ scp initLDTTEST1.ora oracle@Node4:/opt/app/oracle/product/11.2/db_1/dbs
SQL> startup;
ORACLE instance started.
Total System Global Area 9152860160 bytes
Fixed Size 2234056 bytes
Variable Size 6945769784 bytes
Database Buffers 2181038080 bytes
Redo Buffers 23818240 bytes
Database mounted.
Database opened.

Make changes at Listener.ora,tnsnames.ora files on Second Node RAC – Node4

RACNode3> show parameter instance_name
instance_name string LDTTEST1
RACNode3> select count(*) from tab;
4865
RACNode3> show parameter cluster_database
cluster_database boolean TRUE

RACNode4> show parameter instance_name
instance_name string LDTTEST2
RACNode4> select count(*) from tab;
4865
RACNode4> show parameter cluster_database
cluster_database boolean TRUE

Lets make the database Cluster services aware.

[oracle@Node3 dbs]$ srvctl add database -d LDTEST -o /opt/app/oracle/product/11.2/db_1 -p /mnt/data/oradata/LDTEST/spfileLDTTEST.ora
[oracle@Node3 dbs]$ srvctl add instance -d LDTEST -i LDTTEST1 -n Node3
[oracle@Node3 dbs]$ srvctl add instance -d LDTEST -i LDTTEST2 -n Node4
[oracle@Node3 arch]$ /home/oracle/Scripts/crsstat.sh | grep LDTEST
ora.LDTEST.db OFFLINE OFFLINE

Finally  stop/start RAC Databases using srvctl commands

[oracle@Node3 dbs]$ srvctl start database -d LDTEST
[oracle@Node3 dbs]$ $HOME/Scripts/crsstat.sh | grep prod
ora.prod.db ONLINE ONLINE on Node3
[oracle@Node3 dbs]$ srvctl status database -d LDTEST
Instance LDTTEST1 is running on node Node3
Instance LDTTEST2 is running on node Node4

Have a look at alrtlog for any issues reported.

11g |Monitoring DataGuard using Broker Commands

Posted by Sagar Patil

DGMGRL> show database PROD;
Object “prod” was not found

** 11.2.0.2 – You may see errors at dgmgrl if you don’t include database name in quotes

Use : DGMGRL> show database ‘prod’;

1    Check the DG configuration status.
The status of the broker configuration is an aggregated status of all databases and instances in the broker configuration

DGMGRL> show configuration
Configuration – dataguard
Protection Mode: MaxPerformance
Databases:
stdby – Primary database
prod  – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

2   Check the database status  :

DGMGRL> show database  STDBY;
Database – prod
Role:            PHYSICAL STANDBY
Intended State:  APPLY-ON
Transport Lag:   0 seconds
Apply Lag:       1 hour(s) 3 minutes 6 seconds (Note Apply Lag of an hour)
Real Time Query: OFF
Instance(s):
PROD1 (apply instance)
PROD2
Database Status:
SUCCESS

We can also run an SQL to locate the lag between Primary and Standby

set linesize 200;
set pagesize 2000;
COLUMN NAME FORMAT A30;
COLUMN value FORMAT A20;
COLUMN UNIT FORMAT A20;
COLUMN time_computed FORMAT A20;
select name
 , value
 , unit
 , time_computed
 from v$dataguard_stats;
NAME                           VALUE                UNIT                 TIME_COMPUTED
------------------------------ -------------------- -------------------- --------------------
transport lag                  +00 00:22:37         day(2) to second(0)  06/09/2011 12:06:09
 interval
apply lag                      +00 00:22:40         day(2) to second(0)  06/09/2011 12:06:09
 interval
apply finish time              +00 00:00:00.035     day(2) to second(3)  06/09/2011 12:06:09
 interval
estimated startup time         18                   second               06/09/2011 12:06:09

3. Check the monitorable property StatusReport
To identify which database has the failure, you need to go through all of the databases in the configuration one by one.

DGMGRL> show database prod statusreport;
STATUS REPORT
INSTANCE_NAME   SEVERITY ERROR_TEXT

DGMGRL> show database stdby statusreport;
STATUS REPORT
INSTANCE_NAME   SEVERITY ERROR_TEXT

4   Check the monitorable property LogXptStatus
To identify exact log transport errors, we can use monitorable property LogXptStatus

DGMGRL> show database prod logxptstatus
LOG TRANSPORT STATUS
PRIMARY_INSTANCE_NAME STANDBY_DATABASE_NAME               STATUS

5.    Check the monitorable property InconsistentProperties
If you see warning ORA-16714 reported, to identify inconsistent values for property LogArchiveTrace we can use property InconsistentProperties

DGMGRL> SHOW DATABASE prod InconsistentProperties;
INCONSISTENT PROPERTIES
INSTANCE_NAME        PROPERTY_NAME         MEMORY_VALUE         SPFILE_VALUE         BROKER_VALUE

6.    Check the monitorable property InconsistentLogXptProps
To identify the inconsistent values for the redo transport property use monitorable property InconsistentLogXptProps:

DGMGRL> show database prod InconsistentLogXptProps
INCONSISTENT PROPERTIES
INSTANCE_NAME        PROPERTY_NAME         MEMORY_VALUE         SPFILE_VALUE         BROKER_VALUE

Dataguard: Recovering lost broker configuration files: Clear Old Configuration & Add New One

Posted by Sagar Patil

I had a working datagard configuration between 1 primary and 2 physical standby systems. Today I noticed my dataguard configuration files were missing from all servers (Primary/Standby).

DGMGRL> show configuration;
ORA-16532: Data Guard broker configuration does not exist
Configuration details cannot be determined by DGMGRL

Although my dataguard primary-standby were working with existing init values any switchover /failover thru Dataguard single command won’t be possible until broker configuration is in place.

At Both Primary and Secondary :  Set values for parameters fal_server, fal_client,  log_archive_config,
dg_broker_start, log_archive_dest_n

sys@ PRIMARY> show parameter fal
NAME                                 TYPE        VALUE
------------------------------------ ----------- ---------------------------
fal_client                           string      primary
fal_server                           string      secondary

sys@ SECONDARY> show parameter fal;
NAME                                 TYPE        VALUE
———————————— ———– ——————————
fal_client                           string      secondary
fal_server                           string      primary

sys@ PRIMARY/Secondary> alter system set fal_client=”;
System altered.

sys@ PRIMARY/Secondary> alter system set fal_server=”;
System altered.

sys@ PRIMARY/Secondary> alter system set log_archive_dest_2=”;
System altered.

sys@ PRIMARY/SECONDARY> show parameter log_archive_config;
NAME                                 TYPE        VALUE
———————————— ———– ——————————
log_archive_config                   string      dg_config=(secondary,primary)

sys@ PRIMARY/SECONDARY> alter system set log_archive_config=” scope=both;

System altered.
sys@ PRIMARY/SECONDARY> show parameter log_archive_config;
NAME                                 TYPE        VALUE
———————————— ———– ——————————
log_archive_config                   string

sys@ PRIMARY/SECONDARY> show parameter dg_broker_start
NAME                                 TYPE        VALUE
———————————— ———– ——————————
dg_broker_start                      boolean     FALSE

At this point I see there were no archives shipped/ applied at Standby Database

PRIMARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            C:\app\patilsa\product\11.2.0\dbhome_1\oradata\primary
Oldest online log sequence     338
Next log sequence to archive   340
Current log sequence           340

Waited for 10 minutes .. No Archive files shipped from Primary to Standby

sys@ SECONDARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     323
Next log sequence to archive   0
Current log sequence           328

Create a Dataguard Config

Enable Dataguard broker at both systems

sys@ SECONDARY> select instance_name,status from v$instance;
INSTANCE_NAME    STATUS
—————- ————
secondary        MOUNTED

sys@ PRIMARY> select instance_name,status from v$instance;
INSTANCE_NAME    STATUS
—————- ————
primary          OPEN

sys@ PRIMARY> ALTER SYSTEM SET dg_broker_start=TRUE SCOPE=spfile ;
System altered.

sys@ PRIMARY>  show parameter dg_broker
NAME                                 TYPE        VALUE
———————————— ———– ——————————
dg_broker_config_file1               string      C:\APP\PATILSA\PRODUCT\11.2.0\DBHOME_1\DATABASE\DR1PRIMARY.DAT
dg_broker_config_file2               string      C:\APP\PATILSA\PRODUCT\11.2.0\DBHOME_1\DATABASE\DR2PRIMARY.DAT
dg_broker_start                      boolean     TRUE

sys@ SECONDARY> show parameter dg_broker
NAME                                 TYPE        VALUE
———————————— ———– ——————————
dg_broker_config_file1               string      C:\APP\PATILSA\PRODUCT\11.2.0\DBHOME_1\DATABASE\DR1SECONDARY.DAT
dg_broker_config_file2               string      C:\APP\PATILSA\PRODUCT\11.2.0\DBHOME_1\DATABASE\DR2SECONDARY.DAT
dg_broker_start                      boolean     TRUE

DGMGRL> show configuration;
ORA-16532: Data Guard broker configuration does not exist

Configuration details cannot be determined by DGMGRL

DGMGRL> CREATE CONFIGURATION ‘sample’ AS PRIMARY DATABASE IS ‘primary’ CONNECT I
DENTIFIER IS primary;
Configuration “sample” created with primary database “primary”

DGMGRL> show configuration;

Configuration – sample

Protection Mode: MaxPerformance
Databases:
primary – Primary database

Fast-Start Failover: DISABLED

Configuration Status:
DISABLED

DGMGRL> ADD DATABASE ‘secondary’ AS CONNECT IDENTIFIER IS secondary MAINTAINED AS PHYSICAL;
Database “secondary” added
DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary   – Primary database
secondary – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
DISABLED

DGMGRL> enable configuration;
Enabled

DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary   – Primary database
secondary – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

DGMGRL> switchover to secondary;
Performing switchover NOW, please wait…
New primary database “secondary” is opening…
Operation requires shutdown of instance “primary” on database “primary”
Shutting down instance “primary”…
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance “primary” on database “primary”
Starting instance “primary”…
ORACLE instance started.
Database mounted.
Switchover succeeded, new primary is “secondary”

sys@ SECONDARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     367
Next log sequence to archive   369
Current log sequence           369

sys@ PRIMARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            C:\app\patilsa\product\11.2.0\dbhome_1\oradata\primary
Oldest online log sequence     368
Next log sequence to archive   0
Current log sequence           369

Difference between init files :    INITprimary.ORA

Difference between init files :  INITsecondary.ORA

ORA-12514: TNS:listener does not currently know of service requested in connect descriptor at Dataguard Switchover

Posted by Sagar Patil

This is most likely because the special service db_unique_name_DGMGRL has not been registered properly with the listener.

This is one of the requirements when configuring Data Guard broker.

From Oracle Docs :

To enable DGMGRL to restart instances during the course of broker operations, a service with a specific name must be statically registered with the local listener of each instance. The value for the GLOBAL_DBNAME attribute must be set to a concatenation of db_unique_name_DGMGRL.db_domain.

sys@ PRIMARY> show parameter db_uniqu
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_unique_name                       string      primary

sys@ SECONDARY> show parameter db_unique
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_unique_name                       string      secondary

Updated Listener.ora  : I had to add GLOBAL_DBNAME for both databases
SID_LIST_LISTENER =
 (SID_LIST =
 (SID_DESC =
 (SID_NAME = primary)
 (ORACLE_HOME = C:\app\patilsa\product\11.2.0\dbhome_1)
 (GLOBAL_DBNAME = primary_DGMGRL)
 )
 (SID_DESC =
 (SID_NAME = secondary)
 (ORACLE_HOME = C:\app\patilsa\product\11.2.0\dbhome_1)
 (GLOBAL_DBNAME = secondary_DGMGRL)
 )
 )

Dataguard : Activating Standby (Failover) when Primary is Down

Posted by Sagar Patil

In this post, I will show you how easy it is to fail-over on standby dataguard database when primary is not available.

sys@ PRIMARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            C:\app\product\11.2.0\dbhome_1\oradata
Oldest online log sequence     207
Next log sequence to archive   209
Current log sequence           209

sys@ SECONDARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     207
Next log sequence to archive   0
Current log sequence           209

DGMGRL> connect sys/manager@secondary
Connected.
DGMGRL> startup mount;
ORACLE instance started.
Database mounted.

DGMGRL> connect sys/manager@primary
Connected.
DGMGRL> shutdown abort;
ORACLE instance shut down.

DGMGRL> switchover to ‘secondary’;
Performing switchover NOW, please wait…
Error: ORA-01034: ORACLE not available
Error: ORA-16625: cannot reach database “primary”
Failed.
Unable to switchover, primary database is still “primary”

DGMGRL> connect sys/manager@secondary
Connected.
DGMGRL> failover to secondary;
Performing failover NOW, please wait…
Failover succeeded, new primary is “secondary”
DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxPerformance
Databases:
secondary – Primary database
primary   – Physical standby database (disabled)
ORA-16661: the standby database needs to be reinstated
fast-Start Failover: DISABLED
Configuration Status: SUCCESS

Please note Secondary which is now a PRIMARY Database , it’s ARCHIVE log sequence is reset to 1

sys@ SECONDARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     1
Next log sequence to archive   1
Current log sequence           1

If you try and start Primary database (failed over) it won’t be opened by dgmgrl , alrtlog should have following error message

Data Guard: version check completed
Data Guard determines a failover has occurred – this is no longer a primary database
ORA-16649 signalled during: alter database open…

Dataguard : ORA-16826 apply service state is inconsistent when changing mode to MaxProtection

Posted by Sagar Patil

This morning I moved my Standby database from Maxperformance to Maxprotection while it returned ORA-16826

DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxProtection
Databases:
primary   – Primary database
secondary – Physical standby database
Warning: ORA-16826: apply service state is inconsistent with the DelayMins
property
Fast-Start Failover: DISABLED
Configuration Status:
WARNING

When I tried to remove secondary database to correct ORA-16826, I have following error ORA-16627

DGMGRL> remove database ‘secondary’;
Error: ORA-16627: operation disallowed since no standby databases would remain to support protection mode

To correct this issue, I need to move it to MAXPerformance mode

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS  MAXPerformance;
Succeeded.

DGMGRL> remove database secondary;
Removed database “secondary” from the configuration

DGMGRL> ADD DATABASE ‘secondary’ AS CONNECT IDENTIFIER IS secondary MAINTAINED A
S PHYSICAL;
Database “secondary” added

DGMGRL> show database secondary;
Database – secondary
Role:            PHYSICAL STANDBY
Intended State:  OFFLINE
Transport Lag:   (unknown)
Apply Lag:       (unknown)
Real Time Query: OFF
Instance(s): secondary
Database Status:
DISABLED

DGMGRL> enable database secondary;
Enabled.

DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary   – Primary database
secondary – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS  MAXPROTECTION;
Error: ORA-16627: operation disallowed since no standby databases would remain to support protection mode
Failed.

DGMGRL> EDIT DATABASE ‘secondary’ SET PROPERTY ‘LogXptMode’=’SYNC’;
Property “LogXptMode” updated

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS  MAXPROTECTION;
Operation requires shutdown of instance “primary” on database “primary”
Shutting down instance “primary”…
Database closed.
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance “primary” on database “primary”
Starting instance “primary”…

DGMGRL> connect sys/manager@primary
Connected.
DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxProtection
Databases:
primary   – Primary database
secondary – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

Dataguard : EDIT CONFIGURATION (Change Protection Mode)

Posted by Sagar Patil

Following post will give you an idea on Dataguard data protection mode in which you want configuration to run when configuration is enabled. The possible protection modes are:

MAXPROTECTION
MAXAVAILABILITY
MAXPERFORMANCE

Usage Notes :
Before you use EDIT CONFIGURATION command to set the protection mode to either MAXPROTECTION or MAXAVAILABILITY mode, ensure that standby redo log files are configured on a standby database.

* The following table shows the configuration protection modes and the minimum corresponding settings for log transport services:

The following table shows the configuration protection modes and the minimum corresponding
settings for log transport services:
Protection Mode Log Transport Mode Standby Redo Log Files Needed?
MAXPROTECTION SYNC Yes
MAXAVAILABILITY SYNC Yes
MAXPERFORMANCE ARCH or ASYNC Yes for ASYNC

The default protection mode for the configuration is MAXPERFORMANCE.

DGMGRL> show configuration
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary   – Primary database
secondary – Physical standby database

Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

To move the database from MAXPERFORMANCE to MAXPROTECTION or MAXAVAILABILITY we need to configure Log transport mode as SYNC

DGMGRL> EDIT DATABASE ‘primary’ SET PROPERTY ‘LogXptMode’=’SYNC’;
Property “LogXptMode” updated

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS  MAXPROTECTION;
Error: ORA-16627: operation disallowed since no standby databases would remain to support protection mode
Failed.

DGMGRL> EDIT DATABASE ‘secondary’ SET PROPERTY ‘LogXptMode’=’SYNC’;
Error: ORA-16789: standby redo logs not configured
Failed.

Let’s add Standby Redo Logs at Secondary database

sys@ SECONDARY> select * from v$standby_log;
no rows selected

sys@ SECONDARY> ALTER DATABASE ADD STANDBY LOGFILE GROUP 4 SIZE 50M;
ALTER DATABASE ADD STANDBY LOGFILE GROUP 4 SIZE 50M
*
ERROR at line 1:
ORA-01156: recovery or flashback in progress may need access to files

sys@ SECONDARY> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE cancel;
Database altered.

sys@ SECONDARY> ALTER DATABASE ADD STANDBY LOGFILE GROUP 4 SIZE 50M;
Database altered.

sys@ SECONDARY> ALTER DATABASE ADD STANDBY LOGFILE GROUP 5 SIZE 50M;
Database altered.

sys@ SECONDARY> ALTER DATABASE ADD STANDBY LOGFILE GROUP 6 SIZE 50M;
Database altered.

sys@ SECONDARY> select * from v$standby_log;
GROUP# DBID                                        THREAD#  SEQUENCE#      BYTES  BLOCKSIZE       USED ARC STATUS     FIRST_CHANGE# FIRST_TIM NEXT_CHANGE# N
EXT_TIME LAST_CHANGE# LAST_TIME
———- —————————————- ———- ———- ——
4 UNASSIGNED                                        0          0   5242
8800        512        512 YES UNASSIGNED             0                      0
0
5 UNASSIGNED                                        0          0   5242
8800        512        512 YES UNASSIGNED             0                      0
0
6 UNASSIGNED                                        0          0   5242
8800        512        512 YES UNASSIGNED             0                      0
0
3 rows selected.

sys@ SECONDARY> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT from session;
Database altered.

sys@ SECONDARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     145
Next log sequence to archive   0
Current log sequence           147

DGMGRL> EDIT DATABASE ‘secondary’ SET PROPERTY ‘LogXptMode’=’SYNC’;
Property “LogXptMode” updated

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS  MAXPROTECTION;

Operation requires shutdown of instance “primary” on database “primary”
Shutting down instance “primary”…
Database closed.
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance “primary” on database “primary”
Starting instance “primary”…

DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxProtection
Databases:
primary   – Primary database
secondary – Physical standby database

Warning: ORA-16826: apply service state is inconsistent with the DelayMins property How to resolve ORA-16826 error?
Fast-Start Failover: DISABLED
Configuration Status:
WARNING

Summary of DGMGRL Commands with Example

Posted by Sagar Patil


Command Effect
ADD DATABASE Adds a new standby database profile to the existing broker configuration 

DGMGRL> ADD DATABASE ‘secondary’ AS CONNECT IDENTIFIER IS secondary MAINTAINED AS PHYSICAL;

CONNECT Connects to the specified database using the specified username. 

DGMGRL> connect sys/manager@primary
Connected.

CREATE CONFIGURATION Creates a broker configuration and creates and adds a primary database profile to the configuration. 

DGMGRL> CREATE CONFIGURATION ‘sample’ AS PRIMARY DATABASE IS ‘primary’ CONNECT IDENTIFIER IS primary;
Configuration “sample” created with primary database “primary”

DISABLE CONFIGURATION Disables broker management of a configuration so that the configuration and all of its databases are no longer managed by the broker. 

DGMGRL> disable configuration;
Disabled.

DISABLE DATABASE Disables broker management of the named standby database. 

DGMGRL> disable database secondary;
Disabled.

EDIT CONFIGURATION (Protection Mode) Changes the current protection mode setting for the broker configuration 

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS  MAXPerformance;
Succeeded.

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS  MAXPROTECTION;
Operation requires shutdown of instance “primary” on database “primary”
Shutting down instance “primary”…
Database closed.
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance “primary” on database “primary”
Starting instance “primary”…

EDIT DATABASE (Property) Changes the value of a property for the named database 

DGMGRL> EDIT DATABASE ‘primary’ SET PROPERTY ‘ArchiveLagTarget’=200;
Property “ArchiveLagTarget” updated

DGMGRL> EDIT DATABASE ‘primary’ SET PROPERTY ‘StandbyArchiveLocation’=’C:\app\pa
tilsa\product\11.2.0\dbhome_1\oradata’;
Property “StandbyArchiveLocation” updated

EDIT DATABASE (Rename) Changes the name used by the broker to refer to the specified database. This command can only be done when broker management of the database that you are renaming is disabled. 

DGMGRL> EDIT DATABASE ‘secondary’ RENAME TO ‘standby’;
Error: ORA-16602: database must be disabled to perform this operation
Failed.

DGMGRL> disable database secondary;
Disabled.
DGMGRL> EDIT DATABASE ‘secondary’ RENAME TO ‘standby’;
Succeeded.

EDIT DATABASE (State) Changes the state of the specified database.The state in which you want the database to be running. The possible states are:
ONLINE
LOG-TRANSPORT-OFF (primary database only)
LOG-APPLY-OFF (standby database only)
READ-ONLY (physical standby database only)
OFFLINE 

DGMGRL> EDIT DATABASE ‘secondary’ SET STATE=‘READ-ONLY’;
Succeeded.
DGMGRL> EDIT DATABASE ‘secondary’ SET STATE=‘OFFLINE’;
Operation requires shutdown of instance “secondary” on database “secondary”
Shutting down instance “secondary”…
Database closed.
Database dismounted.
ORACLE instance shut down.
DGMGRL> EDIT DATABASE ‘secondary’ SET STATE=’ONLINE’;
Error: ORA-16525: the Data Guard broker is not yet available
— Make sure Standby Database is started and mounted.
DGMGRL> EDIT DATABASE ‘secondary’ SET STATE=’ONLINE’;
Succeeded.

EDIT INSTANCE (AUTO PFILE) Sets the name of the initialization parameter file for the specified instance. 

DGMGRL> EDIT INSTANCE ‘secondary’ ON DATABASE ‘secondary’ SET AUTO PFILE=’initse
condary.ora’;
Instance “secondary” updated

EDIT INSTANCE (Property) Changes the value of a property for the specified instance. 

DGMGRL> EDIT instance ‘primary’ on DATABASE ‘primary’ SET PROPERTY ‘StandbyArchi
veLocation’=’C:\app\patilsa\product\11.2.0\dbhome_1\oradata’;
Property “StandbyArchiveLocation” updated

ENABLE CONFIGURATION Enables broker management of the broker configuration and all of its databases.DGMGRL> enable configuration;
Enabled.
ENABLE DATABASE Enables broker management of the specified database. 

DGMGRL> enable database ‘secondary’;
Enabled.

EXIT Exits the Data Guard command-line interface.
FAILOVER Performs a database failover operation in which the standby database, to which the CLI is currently connected, fails over to the role of primary database.
HELP Displays online help for the Data Guard command-line interface. 

add            Adds a standby database to the broker configuration
connect        Connects to an Oracle database instance
convert        Converts a database from one type to another
create         Creates a broker configuration
disable        Disables a configuration, a database, or fast-start failover
edit           Edits a configuration, database, or instance
enable         Enables a configuration, a database, or fast-start failover
exit           Exits the program
failover       Changes a standby database to be the primary database
help           Displays description and syntax for a command
quit           Exits the program
reinstate      Changes a database marked for reinstatement into a viable standby
rem            Comment to be ignored by DGMGRL
remove         Removes a configuration, database, or instance
show           Displays information about a configuration, database, or instance
shutdown       Shuts down a currently running Oracle database instance
start          Starts the fast-start failover observer
startup        Starts an Oracle database instance
stop           Stops the fast-start failover observer
switchover     Switches roles between a primary and standby database
Use "help <command>" to see syntax for individual commands
QUIT Quits the Data Guard command-line interface.
REMOVE CONFIGURATION Removes the broker configuration including all of its database profiles from the broker configuration file.
REMOVE DATABASE Removes the specified standby database profile from the broker configuration. 

DGMGRL> remove database secondary;
Removed database “secondary” from the configuration

REMOVE INSTANCE Removes knowledge of an instance from an existing database profile in the broker configuration.
SHOW CONFIGURATION Displays information about the broker configuration. 

DGMGRL> SHOW CONFIGURATION
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary   – Primary database
Warning: ORA-16789: standby redo logs not configured
secondary – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
WARNING

SHOW DATABASE Displays information about the specified database. 

DGMGRL> show database secondary;
Database – secondary
Role:            PRIMARY
Intended State:  TRANSPORT-ON
Instance(s):
secondary
Database Status:
SUCCESS

SHOW INSTANCE Displays information about the specified instance. 

DGMGRL> show instance primary;
Instance ‘primary’ of database ‘primary’
Host Name: sagar-pc
Instance Status:
SUCCESS

DGMGRL> show instance secondary;
Instance ‘secondary’ of database ‘secondary’
Host Name: sagar-pc
Instance Status:
SUCCESS

SHUTDOWN Shuts down a currently running Oracle database. 

DGMGRL> connect sys/manager@primary
Connected.
DGMGRL> shutdown abort;
ORACLE instance shut down.

STARTUP Starts an Oracle instance with the same options as SQL*Plus, including mounting and opening a database. 

DGMGRL> connect sys/manager@primary
Connected.
DGMGRL> startup;
ORACLE instance started.
Database mounted.
Database opened.

SWITCHOVER Performs a switchover operation in which the current primary database becomes a standby database, and the specified standby database becomes the primary database. 

DGMGRL> switchover to secondary;
Performing switchover NOW, please wait…
New primary database “secondary” is opening…
Operation requires shutdown of instance “primary” on database “primary”
Shutting down instance “primary”…
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance “primary” on database “primary”
Starting instance “primary”…
Unable to connect to database
ORA-12514: TNS:listener does not currently know of service requested in connectdescriptor Failed. (IGNORE TNS error)
Warning: You are no longer connected to ORACLE.
Please complete the following steps to finish switchover:
start up and mount instance “primary” of database “primary”

The following commands are available: 

add            Adds a standby database to the broker configuration
connect        Connects to an Oracle database instance
convert        Converts a database from one type to another
create         Creates a broker configuration
disable        Disables a configuration, a database, or fast-start failover
edit           Edits a configuration, database, or instance
enable         Enables a configuration, a database, or fast-start failover
exit           Exits the program
failover       Changes a standby database to be the primary database
help           Displays description and syntax for a command
quit           Exits the program
reinstate      Changes a database marked for reinstatement into a viable standby

rem            Comment to be ignored by DGMGRL
remove         Removes a configuration, database, or instance
show           Displays information about a configuration, database, or instance

shutdown       Shuts down a currently running Oracle database instance
start          Starts the fast-start failover observer
startup        Starts an Oracle database instance
stop           Stops the fast-start failover observer
switchover     Switches roles between a primary and standby database

Use “help <command>” to see syntax for individual commands

DGMGRL : ORA-16789: standby redo logs not configured/ORA-16826: apply service state is inconsistent with the DelayMins property

Posted by Sagar Patil

My DGMGRL configuration is returning warnings as below.

DGMGRL> show database primary;
Database – primary
Role:            PRIMARY
Intended State:  TRANSPORT-ON
Instance(s):
primary
Database Warning(s):
ORA-16789: standby redo logs not configured

Database Status:
WARNING

sys@ PRIMARY> SELECT GROUP#,THREAD#,SEQUENCE#,ARCHIVED,STATUS FROM V$STANDBY_LOG;
no rows selected

Let’s go ahead & add Standby Redo Log files at primary

sys@ PRIMARY> select bytes from v$log;
BYTES
———-
52428800
52428800
52428800
3 rows selected.

sys@ PRIMARY> select group#, member from v$logfile;
GROUP#                    MEMBER
——————————————————————————–
1     C:\APP\PATILSA\ORADATA\PRIMARY\REDO01.LOG
2    C:\APP\PATILSA\ORADATA\PRIMARY\REDO02.LOG
3    C:\APP\PATILSA\ORADATA\PRIMARY\REDO03.LOG
3 rows selected.

sys@ PRIMARY> ALTER DATABASE ADD STANDBY LOGFILE GROUP 1 SIZE 50M;
ALTER DATABASE ADD STANDBY LOGFILE GROUP 1 SIZE 50M
*
ERROR at line 1:
ORA-01184: logfile group 1 already exists

My primary database had 3 log file groups originally and I created 3 standby redo log groups using the following commands:

sys@ PRIMARY> ALTER DATABASE ADD STANDBY LOGFILE GROUP 4 SIZE 50M;
Database altered.
sys@ PRIMARY> ALTER DATABASE ADD STANDBY LOGFILE GROUP 5 SIZE 50M;
Database altered.
sys@ PRIMARY> ALTER DATABASE ADD STANDBY LOGFILE GROUP 6 SIZE 50M;
Database altered.

sys@ PRIMARY> select * from v$standby_log;
GROUP# DBID                                        THREAD#  SEQUENCE#      B
YTES  BLOCKSIZE       USED ARC STATUS     FIRST_CHANGE# FIRST_TIM NEXT_CHANGE# NEXT_TIME LAST_CHANGE# LAST_TIME
———- —————————————- ———- ———- ——
4 UNASSIGNED                                        0          0   52428800        512        512 YES UNASSIGNED             0                      0
0
5 UNASSIGNED                                        0          0   52428800        512        512 YES UNASSIGNED             0                      0
0
6 UNASSIGNED                                        0          0   52428800        512        512 YES UNASSIGNED             0                      0
0
3 rows selected.

DGMGRL> show database primary;
Database – primary
Role:            PRIMARY
Intended State:  TRANSPORT-ON
Instance(s):
primary

Database Status:
SUCCESS

If you try and switchover at this point after adding the Standby Redo logs, you will receive a ORA-16826 from the Data Guard Broker.

DGMGRL> switchover to secondary;
Performing switchover NOW, please wait…
Error: ORA-16775: target standby database in broker operation has potential dataloss
Failed.
Unable to switchover, primary database is still “primary”

DGMGRL> show configuration verbose;
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary Primary database
secondary Physical standby database
Warning: ORA-16826: apply service state is inconsistent with the DelayMins
property
Fast-Start Failover: DISABLED
Configuration Status:
WARNING

DGMGRL> remove database secondary;
Removed database “secondary” from the configuration

DGMGRL> ADD DATABASE ‘secondary’ AS CONNECT IDENTIFIER IS secondary MAINTAINED AS PHYSICAL;
Database “secondary” added

DGMGRL> show database secondary;
Database – secondary
Role:            PHYSICAL STANDBY
Intended State:  OFFLINE
Transport Lag:   (unknown)
Apply Lag:       (unknown)
Real Time Query: OFF
Instance(s):
secondary
Database Status:
DISABLED

DGMGRL> enable database secondary;
Enabled.

DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary   – Primary database
secondary – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

DGMGRL> switchover to secondary;
Performing switchover NOW, please wait…
New primary database “secondary” is opening…
Operation requires shutdown of instance “primary” on database “primary”
Shutting down instance “primary”…
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance “primary” on database “primary”
Starting instance “primary”…
Unable to connect to database
ORA-12514: TNS:listener does not currently know of service requested in connectdescriptor Failed.
Warning: You are no longer connected to ORACLE.
Please complete the following steps to finish switchover:
start up and mount instance “primary” of database “primary”

Don’t worry on TNS error ORA-12514: I had to start my PRIMARY database Manually  to fix it.

sys@ SECONDARY> select instance_name, status from v$instance;
INSTANCE_NAME    STATUS
—————- ————
secondary        OPEN
sys@ SECONDARY> select count(*) from tab;
COUNT(*)
———-
4731

sys@ SECONDARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     129
Next log sequence to archive   131
Current log sequence           131

sys@ PRIMARY> select instance_name, status from v$instance;

INSTANCE_NAME    STATUS
—————- ————
primary          MOUNTED

sys@ PRIMARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            C:\app\patilsa\product\11.2.0\dbhome_1\oradata\pr
imary
Oldest online log sequence     131
Next log sequence to archive   0
Current log sequence           133

DGMGRL> connect sys/manager@secondary
Connected.
DGMGRL> show configuration verbose;

Configuration – sample

Protection Mode: MaxPerformance
Databases:
secondary Primary database
primaryPhysical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

DGMGRL> show database ‘primary’ StatusReport;
STATUS REPORT
INSTANCE_NAME   SEVERITY ERROR_TEXT

DGMGRL> show database ‘secondary’ StatusReport;
STATUS REPORT
INSTANCE_NAME   SEVERITY ERROR_TEXT

There are no errors so everything is fine now.

Creating Oracle Dataguard Primary/Standby DB on a Single PC

Posted by Sagar Patil

Generally you don’t have to set up Dataguard on a single machine. But there are occasions when you have to build test environment for destruction and this post is tuned for such opportunity.

There are couple of things you need to worry about when primary & standby are on the same box. This post will address those issues. I have created a new Database called Primary using DBCA. The standby will be addressed as “Secondar” instance here. Log and init parameter files attached.

Files Used : INITprimary.ORA , INITsecondary.ORA

1.    Check the primary database name:

SQL> select name from v$database;
NAME
———
PRIMARY

2.    For physical Dataguard we need to have a Forced logging ON . Let’s check that and enable it.

SQL> select force_logging from v$database;
FOR

NO
SQL> alter database force logging;
Database altered.
SQL> select force_logging from v$database;
FOR

YES

3.    Create a password file

D:\oracle\product\10.2.0\db_1\database>orapwd file=PWDbsl01lab.ora password=manager entries=5
–To verify the new password file
D:\oracle\product\10.2.0\db_1\database>sqlplus /nolog

SQL*Plus: Release 10.2.0.1.0 – Production on Tue Jul 25 22:24:41 2006
Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> connect sys/manager as sysdba
Connected to an idle instance.

4.    Configure a Standby Redo Log

Ensure log file sizes are identical on the primary and standby database
SQL> select group#,thread#,bytes/1024/1024,members from v$log;
GROUP#    THREAD# BYTES/1024/1024    MEMBERS
———- ———- ————— ———-
1          1              50          1
2          1              50          1
3          1              50          1

Determine the appropriate number of standby redo log file groups
The minimal number should be 3+1=4

Verify related database parameters and settings
Maxlogfiles
Maxlogmembers

Create standby redo log file groups
SQL> alter database add standby logfile (‘C:\app\patilsa\oradata\primary\redo01d.log’) size 50M;
Database altered.
SQL> alter database add standby logfile (‘C:\app\patilsa\oradata\primary\redo02d.log’) size 50M;
Database altered.
SQL> alter database add standby logfile (‘C:\app\patilsa\oradata\primary\redo03d.log’) size 50M;
Database altered.

Step 5 Verify the standby redo log file groups were created
SQL> select group#,thread#,sequence#,archived,status from v$standby_log;

GROUP#    THREAD#  SEQUENCE# ARC STATUS
———- ———- ———- — ———-
4          0          0 YES UNASSIGNED
5          0          0 YES UNASSIGNED
6          0          0 YES UNASSIGNED

5.     Set Primary Database Initialization Parameters

SQL> create pfile from spfile;
File created.

–Content of initprimary.ora

primary.__db_cache_size=100663296
primary.__java_pool_size=4194304
primary.__large_pool_size=4194304
primary.__oracle_base='C:\app\patilsa'#ORACLE_BASE set from environment
primary.__pga_aggregate_target=146800640
primary.__sga_target=281018368
primary.__shared_io_pool_size=0
primary.__shared_pool_size=163577856
primary.__streams_pool_size=0
*.audit_file_dest='C:\app\patilsa\admin\primary\adump'
*.audit_trail='db'
*.compatible='11.2.0.0.0'
*.control_files='C:\app\patilsa\oradata\primary\control01.ctl','C:\app\patilsa\product\11.2.0\dbhome_1\oradata\primary\control02.ctl'
*.db_block_size=8192
*.db_domain=''
*.db_name='primary'
*.db_recovery_file_dest='C:\app\patilsa\product\11.2.0\dbhome_1\oradata'
*.db_recovery_file_dest_size=5218762752
*.dg_broker_start=TRUE
*.diagnostic_dest='C:\app\patilsa'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=primaryXDB)'
*.log_archive_format='ARC%S_%R.%T'
*.memory_target=427819008
*.open_cursors=300
*.processes=150
*.remote_login_passwordfile='EXCLUSIVE'
*.undo_tablespace='UNDOTBS1'

SQL> create spfile from pfile;
File created.

6.    Check DB Archiving

SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence     58
Next log sequence to archive   60
Current log sequence           60

If primary is not in Archive mode use following commands to Enable primary DB Archiving

SQL> startup mount;
ORACLE instance started.
Total System Global Area  167772160 bytes
Fixed Size                  1247900 bytes
Variable Size              83887460 bytes
Database Buffers           79691776 bytes
Redo Buffers                2945024 bytes
Database mounted.

SQL> alter database archivelog;
Database altered.
SQL> alter database open;
Database altered.

7.    Create a Backup Copy of the Standby Database

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.

Now Copy all data files/Redos from “”C:\app\patilsa\oradata\primary”  to “C:\app\patilsa\oradata\secondary”

8.    Create a Control File for the Standby Database at Primary

Primary> startup mount
ORACLE instance started.

Total System Global Area  167772160 bytes
Fixed Size                  1247900 bytes
Variable Size              83887460 bytes
Database Buffers           79691776 bytes
Redo Buffers                2945024 bytes
Database mounted.

Primary> Alter Database Create Standby Controlfile as ‘C:\app\patilsa\oradata\secondary\control01.ctl’;
Database altered.

SQL> alter database open;
Database altered.

9.    Create the parameter file for Standby Database : InitStandby.ora

secondary.__db_cache_size=247463936
secondary.__java_pool_size=4194304
secondary.__large_pool_size=4194304
secondary.__oracle_base='C:\app\patilsa'#ORACLE_BASE set from environment
secondary.__pga_aggregate_target=339738624
secondary.__sga_target=507510784
secondary.__shared_io_pool_size=0
secondary.__shared_pool_size=243269632
secondary.__streams_pool_size=0
*.audit_file_dest='C:\app\patilsa\admin\secondary\adump'
*.audit_trail='db'
*.compatible='11.2.0.0.0'
*.control_files='C:\app\patilsa\oradata\secondary\control01.ctl','C:\app\patilsa\flash_recovery_area\secondary\control02.ctl'
*.db_block_size=8192
*.db_domain=''
*.db_name='primary'
*.db_unique_name='secondary'
*.db_recovery_file_dest='C:\app\patilsa\flash_recovery_area'
*.db_recovery_file_dest_size=5218762752
*.db_unique_name='secondary'
*.diagnostic_dest='C:\app\patilsa'
*.dispatchers='(PROTOCOL=TCP) (SERVICE=secondaryXDB)'
*.log_archive_format='ARC%S_%R.%T'
*.memory_target=847249408
*.open_cursors=300
*.processes=150
*.remote_login_passwordfile='EXCLUSIVE'
*.standby_file_management='AUTO'
*.undo_tablespace='UNDOTBS1'
db_file_name_convert=('C:\app\patilsa\oradata\primary' 'C:\app\patilsa\oradata\secondary')
log_file_name_convert=('C:\app\patilsa\oradata\primary' 'C:\app\patilsa\oradata\secondary')
FAL_SERVER=primary
FAL_CLIENT=secondary
STANDBY_FILE_MANAGEMENT=auto 

*.db_name=’primary’ at secondary & *.db_unique_name=’secondary’ # If you don’t set this parameter the standby will fail to start with ORA-01103:

10.    Setup the Environment to Support the Standby Database

Create a windows-based service
D:\oracle\product\10.2.0\db_1>oradim -new -sid secondary -intpwd secondary -startmode manual
Instance created.

Create a password file
D:\oracle\product\10.2.0\db_1>set ORACLE_SID=secondar

C:\data guard\database>orapwd file=PWDsecondary.ora password=secondary entries=5

Configure listeners for the primary and standby database

SID_LIST_LISTENER =
 (SID_LIST =
 (SID_DESC =
 (SID_NAME = CLRExtProc)
 (ORACLE_HOME = C:\app\patilsa\product\11.2.0\dbhome_1)
 (PROGRAM = extproc)
 (ENVS = "EXTPROC_DLLS=ONLY:C:\app\patilsa\product\11.2.0\dbhome_1\bin\oraclr11.dll")
 )
 )
LISTENER =
 (DESCRIPTION_LIST =
 (DESCRIPTION =
 (ADDRESS = (PROTOCOL = TCP)(HOST = sagar-pc)(PORT = 1521))
 (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
 )
 )

Create a server parameter file for a Standby database
D:\oracle\product\10.2.0\db_1\database>set oracle_sid=secondary

D:\oracle\product\10.2.0\db_1\database>sqlplus /nolog

SQL*Plus: Release 10.2.0.1.0 – Production on Wed Jul 26 03:11:04 2006

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

SQL> connect sys/secondary as sysdba
Connected to an idle instance.

SQL> create spfile from pfile;
File created.

SQL> show parameter spfile;

NAME                                 TYPE        VALUE
———————————— ———– ——————————
spfile                               string      C:\APP\PATILSA\PRODUCT\11.2.0\DBHOME_1\DATABASE\SPFILESECONDARY.ORA

11.    Mount the standby DB

SQL> startup nomount;
ORACLE instance started.

Total System Global Area  535662592 bytes
Fixed Size                  1375792 bytes
Variable Size             281018832 bytes
Database Buffers          247463936 bytes
Redo Buffers                5804032 bytes
SQL> alter database mount standby database;
Database altered.

12.    Start Redo Apply

SQL> alter database recover managed standby database disconnect from session;
Database altered.

SQL> show parameter dump

NAME                                 TYPE        VALUE
———————————— ———– ——————————
background_core_dump                 string      partial
background_dump_dest string      c:\app\patilsa\diag\rdbms\secondary\secondary\trace
core_dump_dest                       string      c:\app\patilsa\diag\rdbms\secondary\secondary\cdump
max_dump_file_size                   string      unlimited
shadow_core_dump                     string      none
user_dump_dest                       string      c:\app\patilsa\diag\rdbms\secondary\secondary\trace

13.    Verify the status of primary and standby database

D:\oracle\product\10.2.0\db_1\database>set oracle_sid=primary
SQL> connect / as sysdba
Connected.

SQL> select instance_name, status from v$instance;
INSTANCE_NAME    STATUS
—————- ————
primary      OPEN

D:\oracle\product\10.2.0\db_1\database>set oracle_sid=secondar

SQL> select instance_name,status from v$instance;
INSTANCE_NAME    STATUS
—————- ————
secondar    MOUNTED
Both instance status are OK

Let’s configure a Dataguard BROKER utility for this configuration

  • In order to get Dataguard Broker working make sure you configure a primary & Sandby with spfile
  • Also make sure password file is same for both systems

sys@ PRIMARY>CREATE SPFILE FROM PFILE;
sys@ PRIMARY>SHUTDOWN IMMEDIATE
sys@ PRIMARY>STARTUP
sys@ Secondary>STARTUP MOUNT

To get a clean start, we have to remove all “evidence” left over from possible failed attempts earlier

  1. Make sure the listener is configured with a special entry for DGMGRL
  2. CleanUp archive log destination on the STANDBY database:  [ALTER SYSTEM SET log_archive_dest_1=”;]
  3. Stop DG broker process (DMON) on both, primary and standby database: [ALTER SYSTEM SET dg_broker_start=FALSE SCOPE=spfile SID=’*’;]
  4. Delete any existing DG broker configuration files from the following locations:

$ORACLE_BASE/admin/<db_unique_name> or
$ORACLE_HOME/dbs, namely dr1<db_unique_name>.dat and dr2<db_unique_name>.dat

5. Start the DG broker process (DMON) again on both, the primary and standby databases:
ALTER SYSTEM SET dg_broker_start=TRUE  SCOPE=spfile;

Make sure it is running – check output of SHOW PARAMETER DG

sys@ PRIMARY> ALTER SYSTEM SET dg_broker_start=TRUE SCOPE=spfile ;
System altered.
sys@ PRIMARY> shutdown abort;
ORACLE instance shut down.
sys@ PRIMARY> startup;
ORACLE instance started.
Total System Global Area  426852352 bytes
Fixed Size                  1375060 bytes
Variable Size             318768300 bytes
Database Buffers          100663296 bytes
Redo Buffers                6045696 bytes
Database mounted.
Database opened.
sys@ PRIMARY> SHOW PARAMETER DG
NAME                                 TYPE        VALUE
———————————— ———– ——————————
dg_broker_start                      boolean     TRUE
sys@ PRIMARY> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 – Production With the Partitioning, OLAP, Data Mining and Real Application Testing options

C:\Documents and Settings\patilsa>dgmgrl
DGMGRL> connect sys/manager@primary
Connected.
DGMGRL> CREATE CONFIGURATION ‘sample’ AS PRIMARY DATABASE IS ‘primary’ CONNECT IDENTIFIER IS primary;
Configuration “sample” created with primary database “primary”
DGMGRL> show  configuration
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary – Primary database
Fast-Start Failover: DISABLED
Configuration Status:
DISABLED

DGMGRL> ADD DATABASE ‘secondary’ AS CONNECT IDENTIFIER IS secondary MAINTAINED A
S PHYSICAL;
Error: ORA-16796: one or more properties could not be imported from the database
Failed.

ORA-16796 as Standby database was not started in Recovery mode. I had to run following commands at standby.

sys@ SECONDARY> startup Mount;
sys@ SECONDARY> RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;

DGMGRL> ADD DATABASE ‘secondary’ AS CONNECT IDENTIFIER IS secondary MAINTAINED AS PHYSICAL;
Database “secondary” added
DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary   – Primary database
secondary – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
DISABLED

DGMGRL > ENABLE CONFIGURATION ;
Enabled.

DGMGRL> SHOW CONFIGURATION
Configuration – sample
Protection Mode: MaxPerformance
Databases:
primary   – Primary database
secondary – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

dataguard broker primary after Install.logdataguard broker secondary after Install.log

DGMGRL> show database verbose 'primary';
Database - primary
  Role:            PRIMARY
  Intended State:  TRANSPORT-ON
  Instance(s):   primary
  Properties:
    DGConnectIdentifier             = 'primary'
    ObserverConnectIdentifier       = ''
    LogXptMode                      = 'ASYNC'
    DelayMins                       = '0'
    Binding                         = 'optional'
    MaxFailure                      = '0'
    MaxConnections                  = '1'
    ReopenSecs                      = '300'
    NetTimeout                      = '30'
    RedoCompression                 = 'DISABLE'
    LogShipping                     = 'ON'
    PreferredApplyInstance          = ''
    ApplyInstanceTimeout            = '0'
    ApplyParallel                   = 'AUTO'
    StandbyFileManagement           = 'MANUAL'
    ArchiveLagTarget                = '0'
    LogArchiveMaxProcesses          = '30'
    LogArchiveMinSucceedDest        = '1'
    DbFileNameConvert               = ''
    LogFileNameConvert              = ''
    FastStartFailoverTarget         = ''
    StatusReport                    = '(monitor)'
    InconsistentProperties          = '(monitor)'
    InconsistentLogXptProps         = '(monitor)'
    SendQEntries                    = '(monitor)'
    LogXptStatus                    = '(monitor)'
    RecvQEntries                    = '(monitor)'
    HostName                        = 'sagar-pc'
    SidName                         = 'primary'
    StaticConnectIdentifier         = '(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=Sagar-pc)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=primary_DGMGRL)(INSTANCE_NAME=primary)(SERVER=DEDICATED)))'
    StandbyArchiveLocation          = 'C:\app\patilsa\product\11.2.0\dbhome_1\oradata\primary'
    AlternateLocation               = ''
    LogArchiveTrace                 = '0'
    LogArchiveFormat                = 'ARC%S_%R.%T'
    TopWaitEvents                   = '(monitor)'
Database Status:
SUCCESS
DGMGRL> show database verbose 'secondary';
Database - secondary
  Role:            PHYSICAL STANDBY
  Intended State:  APPLY-ON
  Transport Lag:   0 seconds
  Apply Lag:       19 hours 47 minutes 43 seconds
  Real Time Query: ON
  Instance(s):   secondary
  Properties:
    DGConnectIdentifier             = 'secondary'
    ObserverConnectIdentifier       = ''
    LogXptMode                      = 'ARCH'
    DelayMins                       = '0'
    Binding                         = 'OPTIONAL'
    MaxFailure                      = '0'
    MaxConnections                  = '1'
    ReopenSecs                      = '0'
    NetTimeout                      = '30'
    RedoCompression                 = 'DISABLE'
    LogShipping                     = 'ON'
    PreferredApplyInstance          = ''
    ApplyInstanceTimeout            = '0'
    ApplyParallel                   = 'AUTO'
    StandbyFileManagement           = 'AUTO'
    ArchiveLagTarget                = '0'
    LogArchiveMaxProcesses          = '30'
    LogArchiveMinSucceedDest        = '1'
    DbFileNameConvert               = 'C:\app\patilsa\oradata\primary, C:\app\patilsa\oradata\secondary'
    LogFileNameConvert              = 'C:\app\patilsa\oradata\primary, C:\app\patilsa\oradata\secondary'
    FastStartFailoverTarget         = ''
    StatusReport                    = '(monitor)'
    InconsistentProperties          = '(monitor)'
    InconsistentLogXptProps         = '(monitor)'
    SendQEntries                    = '(monitor)'
    LogXptStatus                    = '(monitor)'
    RecvQEntries                    = '(monitor)'
    HostName                        = 'sagar-pc'
    SidName                         = 'secondary'
    StaticConnectIdentifier         = '(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=sagar-pc)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=secondary_DGMGRL)(INSTANCE_NAME=secondary)(SERVER=DEDICATED)))'
    StandbyArchiveLocation          = 'USE_DB_RECOVERY_FILE_DEST'
    AlternateLocation               = ''
    LogArchiveTrace                 = '0'
    LogArchiveFormat                = 'ARC%S_%R.%T'
    TopWaitEvents                   = '(monitor)'
Database Status:
SUCCESS

DGMGRL> show database 'primary' StatusReport;
STATUS REPORT
       INSTANCE_NAME   SEVERITY ERROR_TEXT

DGMGRL> show database 'secondary' StatusReport;
STATUS REPORT
       INSTANCE_NAME   SEVERITY ERROR_TEXT

No Errors at   StatusReport means everything is fine. Let’s have a look at Archive listings on Primary and Standby

sys@ PRIMARY> alter system switch logfile;
System altered.
sys@ PRIMARY> alter system switch logfile;
System altered.
sys@ PRIMARY> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            C:\app\patilsa\product\11.2.0\dbhome_1\oradata\primary
Oldest online log sequence     102

sys@ PRIMARY> SELECT PROCESS, STATUS, THREAD#, SEQUENCE#, BLOCK#, BLOCKS FROM V$
MANAGED_STANDBY;
PROCESS   STATUS          THREAD#  SEQUENCE#     BLOCK#     BLOCKS
--------- ------------ ---------- ---------- ---------- ----------
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CLOSING               1         93          1       1361
ARCH      CLOSING               1         97          1          1
ARCH      CLOSING               1         94          1          1
ARCH      CLOSING               1         95          1         11
ARCH      CLOSING               1         99          1         26
ARCH      CLOSING               1         94          1          1
ARCH      CLOSING               1         96          1          2
ARCH      CLOSING               1         98       2049        510
ARCH      CLOSING               1         98          1       2558
ARCH      CLOSING               1         96          1          2
ARCH      CLOSING               1         99          1         26
ARCH      CONNECTED             0          0          0          0
ARCH      CLOSING               1        100          1        136
ARCH      CLOSING               1        101          1         17
ARCH      CLOSING               1        102          1          8
ARCH      CLOSING               1        101          1         17
ARCH      CLOSING               1        102          1          8
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CLOSING               1        103          1          8
ARCH      CLOSING               1        103          1          8
ARCH      CONNECTED             0          0          0          0
ARCH      CLOSING               1         97          1          1
ARCH      CLOSING               1         95          1         11
ARCH      CONNECTED             0          0          0          0
ARCH      CLOSING               1         93          1       1361
30 rows selected.

sys@ SECONDARY> SELECT PROCESS, STATUS, THREAD#, SEQUENCE#, BLOCK#, BLOCKS FROM
V$MANAGED_STANDBY;
PROCESS   STATUS          THREAD#  SEQUENCE#     BLOCK#     BLOCKS
--------- ------------ ---------- ---------- ---------- ----------
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
ARCH      CONNECTED             0          0          0          0
MRP0 WAIT_FOR_LOG 1 104 0 0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
RFS       IDLE                  0          0          0          0
43 rows selected.

See tail of Broker log files on what broker does when switchover happens    : drcprimary.log & drcsecondary.log

DGMGRL> show configuration;
Configuration – sample
Protection Mode: MaxPerformance
Databases:
secondary – Primary database
primary   – Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS

DGMGRL> show database ‘secondary’ StatusReport;
STATUS REPORT
INSTANCE_NAME   SEVERITY ERROR_TEXT

DGMGRL> show database ‘primary’ StatusReport;
STATUS REPORT
INSTANCE_NAME   SEVERITY ERROR_TEXT

Let’s carry another switchover to make sure it’s all OK.


DGMGRL> show database verbose ‘secondary’;Database – secondary

Role:            PHYSICAL STANDBY
Intended State:  APPLY-ON
Transport Lag:   0 seconds
Apply Lag:       19 hours 47 minutes 43 seconds
Real Time Query: ON
Instance(s):
secondary

Properties:
DGConnectIdentifier             = ‘secondary’
ObserverConnectIdentifier       = ”
LogXptMode                      = ‘ARCH’
DelayMins                       = ‘0’
Binding                         = ‘OPTIONAL’
MaxFailure                      = ‘0’
MaxConnections                  = ‘1’
ReopenSecs                      = ‘0’
NetTimeout                      = ’30’
RedoCompression                 = ‘DISABLE’
LogShipping                     = ‘ON’
PreferredApplyInstance          = ”
ApplyInstanceTimeout            = ‘0’
ApplyParallel                   = ‘AUTO’
StandbyFileManagement           = ‘AUTO’
ArchiveLagTarget                = ‘0’
LogArchiveMaxProcesses          = ’30’
LogArchiveMinSucceedDest        = ‘1’
DbFileNameConvert               = ‘C:\app\patilsa\oradata\primary, C:\app\pa
tilsa\oradata\secondary’
LogFileNameConvert              = ‘C:\app\patilsa\oradata\primary, C:\app\pa
tilsa\oradata\secondary’
FastStartFailoverTarget         = ”
StatusReport                    = ‘(monitor)’
InconsistentProperties          = ‘(monitor)’
InconsistentLogXptProps         = ‘(monitor)’
SendQEntries                    = ‘(monitor)’
LogXptStatus                    = ‘(monitor)’
RecvQEntries                    = ‘(monitor)’
HostName                        = ‘GBDXP04184’
SidName                         = ‘secondary’
StaticConnectIdentifier         = ‘(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST
=GBDXP04184)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=secondary_DGMGRL)(INSTANCE_N
AME=secondary)(SERVER=DEDICATED)))’
StandbyArchiveLocation          = ‘USE_DB_RECOVERY_FILE_DEST’
AlternateLocation               = ”
LogArchiveTrace                 = ‘0’
LogArchiveFormat                = ‘ARC%S_%R.%T’
TopWaitEvents                   = ‘(monitor)’

Database Status:
SUCCESS

Dataguard : Applying CPU(Cirital Patch Update) to Dataguard Environment

Posted by Sagar Patil

I have One Primary database and 2 Physical Standby Databases under RHEL 5. Here is a process to patch them with JULY CPU update “Critical Patch Update Release 10.2.0.4 for UNIX Released July 13, 2010”

At Primary Site
SQL> show parameter dg_broker_start

NAME                                 TYPE        VALUE
———————————— ———– ——————————
dg_broker_start                      boolean     TRUE

SQL> alter system switch logfile;
System altered.

Make sure the Logs are shipped and both Standby Database and applied. Following SQLs could be used to locate the log sequences on all databases.

Locate Primary Log Sequence :   Select Archived_Seq# From V$Archive_Dest_Status Where Dest_Id=1;

Locate Standby Log Sequence :  Select Archived_Seq#, Applied_Seq# From V$Archive_Dest_Status Where Dest_Id=2;

Locate DR Log Sequence :   Select Archived_Seq#, Applied_Seq# From V$Archive_Dest_Status Where Dest_Id=3;

Find out existing Patch details using following queries :

SELECT comments, action_time, id “PATCH_NUMBER”, version FROM sys.registry$history WHERE action = ‘CPU’;

COMMENTS ACTION_TIME PATCH_NUMBER VERSION
view recompilation 42:39.2 6452863
view recompilation 59:20.3 6452863
view recompilation 23:58.7 6452863
view recompilation 56:19.9 6452863

SELECT comments, action_time, id PATCH_NUMBER, version FROM sys.registry$history

COMMENTS ACTION_TIME PATCH_NUMBER VERSION
Upgraded from 10.2.0.1.0 40:28.8 10.2.0.4.0
CPUApr2009 46:06.0 4 10.2.0.4
view recompilation 42:39.2 6452863
CPUOct2009 56:35.7 6 10.2.0.4
view recompilation 59:20.3 6452863
CPUJan2010 01:47.4 6 10.2.0.4
view recompilation 23:58.7 6452863

Backup Primary database  & then shutdown Primary Database

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

Take TAR backup of ORACLE_HOME

$tar -czvf ORACLE_BKUP.tgz $ORACLE_HOME

At Standby & DR Site
[oracle@Standby bdump]$ sqlplus /nolog

SQL*Plus: Release 10.2.0.4.0 – Production on Mon Mar 29 18:26:46 2010

Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.

SQL> conn / as sysdba
Connected.
SQL> select NAME,open_mode,GUARD_STATUS,DATABASE_ROLE from v$database;

NAME      OPEN_MODE  GUARD_S DATABASE_ROLE
——— ———- ——- —————-
ROD  MOUNTED    NONE    PHYSICAL STANDBY

SQL> shutdown immediate;
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down. (Took 5 mins + to shutdown)

Take TAR backup of Standby ORACLE_HOME

$tar -czvf ORACLE_BKUP.tgz $ORACLE_HOME

[oracle@ ~]$ lsnrctl stop LISTENER
[oracle@ ~]$ ps -ef | grep oracle
oracle    3861  3802  0 18:45 pts/0    00:00:00 -bash
oracle    7309  7287  0 20:30 pts/1    00:00:01 -bash
oracle    8907  7309  0 21:19 pts/1    00:00:00 ps -ef
oracle    8908  7309  0 21:19 pts/1    00:00:00 grep oracle

cd $ORACLE_HOME/patches
[oracle@patches]$ ls -lrt
total 12
drwxr-xr-x  5 oracle oracle 4096 Feb 15  2010 9119284
drwxr-xr-x 37 oracle oracle 4096 Feb 15  2010 9119226
drwxr-xr-x 42 oracle oracle 4096 Jul  6 16:53 9655017

[oracle@patches]$ cd 9655017

[oracle@patches]$ $ORACLE_HOME/OPatch/opatch napply -skip_subset -skip_duplicate

Stop here go back to Primary now

oracle@ ~]$ lsnrctl stop LISTENER
[oracle@ ~]$ ps -ef | grep oracle
oracle    3861  3802  0 18:45 pts/0    00:00:00 -bash
oracle    7309  7287  0 20:30 pts/1    00:00:01 -bash
oracle    8907  7309  0 21:19 pts/1    00:00:00 ps -ef
oracle    8908  7309  0 21:19 pts/1    00:00:00 grep oracle

cd $ORACLE_HOME/patches
[oracle@patches]$ ls -lrt
total 12
drwxr-xr-x  5 oracle oracle 4096 Feb 15  2010 9119284
drwxr-xr-x 37 oracle oracle 4096 Feb 15  2010 9119226
drwxr-xr-x 42 oracle oracle 4096 Jul  6 16:53 9655017

[oracle@patches]$ cd 9655017

[oracle@patches]$ $ORACLE_HOME/OPatch/opatch napply -skip_subset -skip_duplicate

For each database instance running on the Oracle home being patched, connect to the database using SQL*Plus.

cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle.sql cpu apply
SQL> — Execute the next statement only if this is the first 10.2.0.4 CPU applied in the Oracle home.
SQL> @utlrp.sql
SQL> QUIT

Run the view recompilation script. Note that this script is run with the database in upgrade mode, which restricts connections as SYSDBA.

cd $ORACLE_HOME/cpu/view_recompile
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> SHUTDOWN IMMEDIATE
SQL> STARTUP UPGRADE
SQL> @view_recompile_jan2008cpu.sql
SQL> SHUTDOWN;
SQL> STARTUP;
SQL> QUIT

Startup Primary Listener

At Standby & DR Site

SQL> startup nomount;
ORACLE instance started.
Total System Global Area 2147483648 bytes
Fixed Size 2030296 bytes
Variable Size 503317800 bytes
Database Buffers 1627389952 bytes
Redo Buffers 14745600 bytes

SQL> show parameter dg_broker_start
NAME TYPE VALUE
———————————— ———– ——————————
dg_broker_start boolean TRUE

SQL> alter database mount standby database;
Database altered.

SQL> recover managed standby database disconnect from session;
Media recovery complete.

Startup Listeners and monitor alrtlogs at Stanby servers for Log Recovery

Redo Shipping Client Connected as PUBLIC
— Connected User is Valid
RFS[3]: Assigned to RFS process 7804
RFS[3]: Identified database type as ‘physical standby’
RFS[3]: Archived Log: ‘/u06/oradata/prod/arch/1_65479_684936861.arc’
Wed Sep 29 22:08:08 2010
Media Recovery Log /u06/oradata/prod/arch/1_65469_684936861.arc
Wed Sep 29 22:08:08 2010
RFS[1]: Archived Log: ‘/u06/oradata/prod/arch/1_65477_684936861.arc’
Wed Sep 29 22:08:09 2010
Media Recovery Log /u06/oradata/prod/arch/1_65470_684936861.arc
Media Recovery Log /u06/oradata/prod/arch/1_65471_684936861.arc
Media Recovery Log /u06/oradata/prod/arch/1_65472_684936861.arc
Media Recovery Log /u06/oradata/prod/arch/1_65473_684936861.arc
Media Recovery Log /u06/oradata/prod/arch/1_65474_684936861.arc
Media Recovery Log /u06/oradata/prod/arch/1_65475_684936861.arc
Media Recovery Log /u06/oradata/prod/arch/1_65476_684936861.arc

Finally run following commands to make sure logs are appied at standbydatabases:

Locate Primary Log Sequence :   Select Archived_Seq# From V$Archive_Dest_Status Where Dest_Id=1;

Locate Standby Log Sequence :  Select Archived_Seq#, Applied_Seq# From V$Archive_Dest_Status Where Dest_Id=2;

Locate DR Log Sequence :   Select Archived_Seq#, Applied_Seq# From V$Archive_Dest_Status Where Dest_Id=3;

Sample log files attached  opatch_history-1.txt & opatch.log

Dataguard: Shell script to List Archive gap between Primary & Standby

Posted by Sagar Patil

I was looking for a NAGIOS monitoring script which will list the archive gap between Primary and 2 of my Standby databases.

http://www.oracledbasupport.co.uk/wp-content/uploads/2010/09/dg_archive_gap_listing-1.txt

When compiled the script will return results as below

[oracle@scripts]$ ./dg_archive_gap.sh
Primary_Arc_No=61077 but Stby_Arc_No=61066

[oracle@scripts]$ ./dg_archive_gap.sh
Primary_Arc_No=61088 but Stby_Arc_No=61047
Stby_Arc_No=61067 but Stby_Apply_No=61047
Dr_Arc_No=61067 but Dr_Apply_No=61047

If you have multiple standby databases , please make sure you Add SQL stmts with “Dest_Id=X” , here 1 (Primary), 2 (Standby) & 3 (DR standby)

#!/bin/bash
#set Oracle environment for Sql*Plus
export ORACLE_HOME=/u01/app/oracle/product/10.2.0
export ORACLE_SID=prod
export PATH=$PATH:$ORACLE_HOME/bin

#set working directory. script is located here..
cd /home/oracle/scripts

#Problem statement is constructed in message variable
MESSAGE=””

#hostname of the primary DB.. used in messages..
HOST_NAME=`hostname`

#Get Data guard information to Unix shell variables…

MESSAGE=`
sqlplus -s sys/sys as sysdba <<EOSQL
set pages 0 feed off
set serverout on size 1000000;
Var Rc Number;
Declare
Local_Arc             Number:=0;
Stby_Arc              Number:=0;
Dr_Arc                Number:=0;
Stby_Apply            Number:=0;
Dr_Apply              Number:=0;
Msg                   Varchar2(4000):=”;
Begin
:Rc:=0;
Select Archived_Seq# Into Local_Arc
From V\\$Archive_Dest_Status Where Dest_Id=1;
Select Archived_Seq#, Applied_Seq#
Into Stby_Arc, Stby_Apply
From V\\$Archive_Dest_Status Where Dest_Id=2;
Select Archived_Seq#, Applied_Seq#
Into Dr_Arc, Dr_Apply
From V\\$Archive_Dest_Status Where Dest_Id=3;

If Local_Arc > Stby_Arc + 10 Then
Msg:=Msg||’Primary_Arc_No=’||Local_Arc||’ but Stby_Arc_No=’||Stby_Arc||Chr(10) || ‘–‘;
:Rc:=1;
End If;

If Stby_Arc > Stby_Apply + 10 Then
Msg:=Msg||’Stby_Arc_No=’||Stby_Arc||’ but Stby_Apply_No=’||Stby_Apply||Chr(10) || ‘–‘;
:Rc:=1;
End If;

If :Rc != 0 Then
Dbms_Output.Put_Line(Msg);
End If;

If Dr_Arc > Dr_Apply + 10 Then
Msg:=Msg||’Dr_Arc_No=’||Dr_Arc||’ but Dr_Apply_No=’||Dr_Apply||Chr(10)|| ‘–‘;
:Rc:=1;
End If;

If :Rc != 0 Then
Dbms_Output.Put_Line(Msg);
End If;
End;
/
exit :rc
EOSQL`
echo $MESSAGE
if [ $? -ne 0 ]; then
MESSAGE=”Error on $HOST_NAME Standby -log APPLY- service!\n${MESSAGE}\nThis problem may cause the archive directories to get full!!!\n\n”
echo -e $MESSAGE
fi

Script to Auto Start Primary/Standby Database under Linux

Posted by Sagar Patil

The following represents the Oracle recommended method for automating database startup and shutdown of Oracle 10g instances.

Once the instance is created, edit the “/etc/oratab” file setting the restart flag for each instance to ‘Y’.
dev:/u01/app/oracle/product/10.2.0:N
Change N to Y
dev:/u01/app/oracle/product/10.2.0:Y

Next, create a file called “/etc/init.d/dbora” as the root user, containing the following.

#!/bin/sh
# chkconfig: 345 99 10
# description: Oracle auto start-stop script.
#
# Set ORA_HOME to be equivalent to the $ORACLE_HOME
# from which you wish to execute dbstart and dbshut;
#
# Set ORA_OWNER to the user id of the owner of the
# Oracle database in ORA_HOME.

ORA_HOME=/u01/app/oracle/product/10.2.0/
ORA_OWNER=oracle

if [ ! -f $ORA_HOME/bin/dbstart ]
then
echo “Oracle startup: cannot start”
exit
fi

case “$1” in
‘start’)
# Start the Oracle databases:
# The following command assumes that the oracle login
# will not prompt the user for any values
su – $ORA_OWNER -c “$ORA_HOME/bin/dbstart $ORA_HOME”
;;
‘stop’)
# Stop the Oracle databases:
# The following command assumes that the oracle login
# will not prompt the user for any values
su – $ORA_OWNER -c “$ORA_HOME/bin/dbshut $ORA_HOME”
;;
esac

set the chmod command to set the privileges to 750
chmod 750 /etc/init.d/dbora

Associate the dbora service with the appropriate run levels and set it to auto-start using the following command.
chkconfig –add dbora

The default TNS listener (LISTENER) will be started by dbshut/dbstart scripts.

If you are using customized listener name like LISTENER_TEST, LISTENER_PRE then please edit dbshut and dbstart lines as below

— At DBSHUT
$ORACLE_HOME_LISTNER/bin/lsnrctl LISTENER stop    “Change to”
$ORACLE_HOME_LISTNER/bin/lsnrctl LISTENER_TEST stop

–AT DBSTART
$ORACLE_HOME_LISTNER/bin/lsnrctl LISTENER start    “Change to”
$ORACLE_HOME_LISTNER/bin/lsnrctl LISTENER_TEST start

The relevant instances should now startup/shutdown automatically at system startup/shutdown

Views Relevant to Oracle Data Guard

Posted by Sagar Patil
LOGICAL DBA_LOGSTDBY_EVENTS Contains information about the activity of the logical standby database system. It can be used to determine the cause of failures that occur when SQL Apply is applying redo to logical standby databases.
LOGICAL DBA_LOGSTDBY_LOG Shows the log files registered for logical standby databases.
LOGICAL DBA_LOGSTDBY_NOT_UNIQUE Identifies tables that have no primary and no non-null unique indexes.
LOGICAL DBA_LOGSTDBY_PARAMETERS Contains the list of parameters used by SQL apply.
LOGICAL DBA_LOGSTDBY_PROGRESS Describes the progress of SQL Apply on the logical standby database.
LOGICAL DBA_LOGSTDBY_SKIP Lists the tables that will be skipped by SQL Apply.
LOGICAL DBA_LOGSTDBY_SKIP_TRANSACTION Lists the skip settings chosen.
  DBA_LOGSTDBY_UNSUPPORTED Identifies the schemas and tables (and columns in those tables) that contain unsupported datatypes. Use this view when you are preparing to create a logical standby database.
PHYSICAL/LOGICAL V$ARCHIVE_DEST

Describes, for the current instance, all of the destinations in the Data Guard configuration, including each destination’s current value, mode, and status.
Note: The information in this view does not persist across an instance shutdown.

PHYSICAL/LOGICAL V$ARCHIVE_DEST_STATUS

Displays runtime and configuration information for the archived redo log destinations.
Note: The information in this view does not persist across an instance shutdown.

PHYSICAL/LOGICAL V$ARCHIVE_GAP Displays information to help you identify a gap in the archived redo log files.
PHYSICAL/LOGICAL V$ARCHIVED_LOG Displays archive redo log information from the control file, including names of the archived redo log files.
PHYSICAL/LOGICAL V$DATABASE Provides database information from the control file.
PHYSICAL/LOGICAL V$DATABASE_INCARNATION Displays information about all database incarnations. Oracle Database creates a new incarnation whenever a database is opened with the RESETLOGS option. Records about the current and the previous incarnation are also contained in the V$DATABASE view.
PHYSICAL/LOGICAL V$DATAFILE Provides datafile information from the control file.
PHYSICAL/LOGICAL V$DATAGUARD_CONFIG Lists the unique database names defined with the DB_UNIQUE_NAME and LOG_ARCHIVE_CONFIG initialization parameters.
PHYSICAL/LOGICAL V$DATAGUARD_STATUS Displays and records events that would typically be triggered by any message to the alert log or server process trace files.
PHYSICAL/LOGICAL V$LOG Contains log file information from the online redo log files.
PHYSICAL/LOGICAL V$LOGFILE Contains information about the online redo log files and standby redo log files.
PHYSICAL/LOGICAL V$LOG_HISTORY Contains log history information from the control file.
PHYSICAL V$MANAGED_STANDBY

Displays current status information for Oracle database processes related to physical standby databases.
Note: The information in this view does not persist across an instance shutdown.

PHYSICAL/LOGICAL V$STANDBY_LOG Contains log file information from the standby redo log files.

Manually resolve archive log gap/ Recover Standby / Manual Standby recovery

Posted by Sagar Patil

The primary and standby database have the same filesystem layout, i.e. archive redo is at same place on both servers.

References: Metalink note 150214.1 – Synchronization of primary and standby database due to log transfer gap
Identify the missing log sequence numbers by running this SQL on the STANDBY database:

SQL> select thread#, low_sequence#, high_sequence#  from v$archive_gap;

Identify the filenames of the missing logs by running this SQL on the PRIMARY database:

SQL>SELECT   NAME
FROM   V$ARCHIVED_LOG
WHERE   DEST_ID = 1 AND SEQUENCE# BETWEEN &Low_Sequence AND &High_Sequence;

1. Cancel managed recovery in the STANDBY database

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;

2. Copy the missing archive redo logs from primary to standby server.

This can be done using any available O/S utility. If you use ftp make sure the transfer is done in binary mode. We assume here that the redo files are going to the same location on the standby server.

3. Recover the STANDBY database

SQL> RECOVER AUTOMATIC STANDBY DATABASE;

When all logs have been applied Oracle will ask for the next, as yet unarchived, redo log. At this point cancel the recovery, e.g.

ORA-00279: change 2668131127884 generated at 12/16/2004 07:45:58 needed forthread 2
ORA-00289: suggestion : /oraarchive/standby/arc2_00000001992.log
ORA-00280: change 2668131127884 for thread 2 is in sequence #1992
ORA-00278: log file ‘/oraarchive/standby/arc2_00000001992.log’no longer needed for this recovery
ORA-00308: cannot open archived log ‘/oraarch01/smp/arch/arc2_00000001992.log’
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
Additional information: 3
Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
CANCEL

4. Put the STANDBY database back into managed recovery mode:

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT;

5. Check that the archive redo gaps have been resolved byt running thie SQL on the STANDBY database:

SQL> select thread#, low_sequence#, high_sequence# from v$archive_gap;

How to Recover Standby from Primary Database

Posted by Sagar Patil

— Server A – Is a Live Server
— Server B – Is a Physical Standby Server
Both were configured for a data guard

One fine day a standby blows away and we need to recreate it from Live

At Live Server A

1. There is no need but if you want please disable Dataguard Broker

SQLPLUS> show parameter dg_broker_start

NAME TYPE VALUE
———————————— ———– ——————————
dg_broker_start boolean TRUE

SQLPLUS> alter system set dg_broker_start=FALSE

DGMGRL> show configuration verbose;

Configuration
Name: odba
Enabled: YES
Protection Mode: MaxPerformance
Fast-Start Failover: DISABLED
Databases:
catp_hongkong – Primary database
catp_morocco – Physical standby database (disabled)

2. Put current database into a HOT backup mode.
Backup Datafiles, Controlfiles and Redo logs

select ‘alter tablespace ‘|| tablespace_name ||’ begin backup;’ from dba_tablespaces;

select ‘cp ‘|| name || ‘ /oracle5/orabkup/ ‘ from v$datafile/v$controlfile/v$logfile

alter database create standby controlfile as ‘/oracle5/orabkup/control_sndby.ctl’

select ‘alter tablespace ‘|| tablespace_name ||’ end backup;’ from dba_tablespaces;


Copy files from backup set on a standby database.

At Standby Server B

3. Please make sure following parameters are set on a Physical Standby database init.ora file

Add 2 parameters at init.ora on standby only
standby_file_management=auto
standby_archive_dest=’f:\oracle\oradata\standby’


4. Recover Standby Database Now

SQL> connect / as sysdba
Connected.

SQL> archive log list;
Database log mode Archive Mode
Automatic archival Enabled
Archive destination /oracle1/oradata/catp
Oldest online log sequence 199166
Next log sequence to archive 199169
Current log sequence 199169
SQL> show parameter dg_broker_start

NAME TYPE VALUE
———————————— ———– ——————————
dg_broker_start boolean FALSE

SQL> alter system set dg_broker_start=TRUE scope=both;
System altered.

SQL> startup nomount;
ORACLE instance started.

Total System Global Area 2147483648 bytes
Fixed Size 2030296 bytes
Variable Size 503317800 bytes
Database Buffers 1627389952 bytes
Redo Buffers 14745600 bytes

SQL> show parameter dg_broker_start

NAME TYPE VALUE
———————————— ———– ——————————
dg_broker_start boolean TRUE

SQL> recover managed standby database disconnect from session;
ORA-01507: database not mounted

SQL> alter database mount standby database;

Database altered.

SQL> recover managed standby database disconnect from session;
Media recovery complete.

Monitor Alrt log for messages for ARCHIVE Success

Media Recovery Waiting for thread 1 sequence 199169
Fetching gap sequence in thread 1, gap sequence 199169-199177
Tue Jun 5 15:54:45 2007
Redo Shipping Client Connected as PUBLIC
— Connected User is Valid
RFS[2]: Assigned to RFS process 3397
RFS[2]: Identified database type as ‘physical standby’
RFS[2]: No standby redo logfiles created
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199179_1_528884944.arc’
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199169_1_528884944.arc’
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199170_1_528884944.arc’
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199171_1_528884944.arc’
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199172_1_528884944.arc’
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199173_1_528884944.arc’
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199174_1_528884944.arc’
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199175_1_528884944.arc’
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199176_1_528884944.arc’
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199177_1_528884944.arc’
Tue Jun 5 15:55:15 2007
Media Recovery Log /oracle1/oradata/catp/catp_0000199169_1_528884944.arc
Media Recovery Log /oracle1/oradata/catp/catp_0000199170_1_528884944.arc
Media Recovery Log /oracle1/oradata/catp/catp_0000199171_1_528884944.arc
Media Recovery Log /oracle1/oradata/catp/catp_0000199172_1_528884944.arc
Media Recovery Log /oracle1/oradata/catp/catp_0000199173_1_528884944.arc
Media Recovery Log /oracle1/oradata/catp/catp_0000199174_1_528884944.arc
Media Recovery Log /oracle1/oradata/catp/catp_0000199175_1_528884944.arc
Media Recovery Log /oracle1/oradata/catp/catp_0000199176_1_528884944.arc
Media Recovery Log /oracle1/oradata/catp/catp_0000199177_1_528884944.arc
Media Recovery Waiting for thread 1 sequence 199178
Fetching gap sequence in thread 1, gap sequence 199178-199178
Tue Jun 5 15:55:18 2007
RFS[2]: Archived Log: ‘/oracle1/oradata/catp/catp_0000199178_1_528884944.arc’

Note :
You don’t need to use “recover managed standby database disconnect from session;” if you have used ” alter system set dg_broker_start=TRUE scope=both;”

You can check v$archived_log and v$dataguard_status views to find out logs applied and errors if any respectively.

1. Logs Applied on Standby
Select sequence#, applied from v$archived_log where STANDBY_DEST=’YES’ order by sequence#;
OR
select sequence#, applied ,completion_time from v$archived_log where STANDBY_DEST=’YES’ order by 3 desc

2. Error message at Dataguard
Select to_char(TIMESTAMP, ‘dd-mon-yyyy hh24:mm:ss’), MESSAGE from v$dataguard_status;

Sometimes Oracle throws error messages at ALRTLOG – Failed to request gap sequence

Reason : DG FAL failed to copy archive files from PRIMARY to STANDBY

Media Recovery Waiting for thread 1 sequence 84730
Fetching gap sequence in thread 1, gap sequence 84730-85042
FAL[client]: Trying FAL server: (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=zagreb)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=bfp_zagreb_XPT.yell)(SERVER=dedicated)))
Mon Nov 12 15:06:45 2007
Completed: ALTER DATABASE RECOVER managed standby database d
Mon Nov 12 15:11:45 2007
FAL[client]: Failed to request gap sequence for thread 1 gap sequence 84730-85042
FAL[client]: All defined FAL servers have been attempted.

Fix:

SQL> recover managed standby database disconnect from session;
Media recovery complete.
SQL> Select sequence#, applied from v$archived_log where STANDBY_DEST=’YES’ order by sequence#;

no rows selected

SQL> recover standby database;
ORA-01153: an incompatible media recovery is active

STOP EARLIER recovery session

SQL> recover managed standby database cancel;
Media recovery complete.

SQL> recover standby database;
ORA-00279: change 1967405177515 generated at 11/08/2007 10:06:55 needed for
thread 1
ORA-00289: suggestion : /oracle1/oradata/bfp/bfp_1_569156389_0000084730.arc
ORA-00280: change 1967405177515 for thread 1 is in sequence #84730

Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
auto

This will end up as

ORA-00308: cannot open archived log
‘/oracle1/oradata/bfp/bfp_1_569156389_0000084800.arc’
ORA-27037: unable to obtain file status
HPUX-ia64 Error: 2: No such file or directory
Additional information: 3

SQL> shutdown immediate;
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.

Rman Script to Create Standby From Live Server
connect target rman/obst@live
connect auxiliary sys/ams@standby
connect rcvcat rmanc/obss@rcvcat
Run {
allocate auxiliary channel tp1 type “sbt_tape”;
allocate auxiliary channel tp2 type “sbt_tape”;
send ‘NSR_ENV=(NSR_SERVER=bsbk10,NSR_DATA_VOLUME_POOL=Default,NSR_CLIENT=miles)’;
duplicate target database for standby dorecover nofilenamecheck;
release channel tp1;
release channel tp2;
}

Above script will return an error
ORA-01830: date format picture ends before converting entire input string

Add a line
Run {
set until time “to_date(‘Dec 11 2007 06:30:09′,’Mon DD YYYY HH24:MI:SS’)”;

Restore Archivelogs which were backed up thru rman

connect rcvcat rmanc/obseq@rcvcat
connect target rmant/obus@live
run {
allocate channel tp1 type “sbt_tape”
parms ‘ENV=(NSR_SERVER=bsbk10,NSR_DATA_VOLUME_POOL=Default)’;
allocate channel tp2 type “sbt_tape”
parms ‘ENV=(NSR_SERVER=bsbk10,NSR_DATA_VOLUME_POOL=Default)’;
set archivelog destination to ‘f:\oracle\oradata\live’;
restore archivelog from logseq 111094 until logseq 111111;
release channel tp1;
release channel tp2;

Run RMAN script to create log as
rman cmdfile=create_standby.rman msglog=create_standby.log

Oracle Standby Recovery Rate Monitoring

Posted by Sagar Patil

Why standby periodically lags during the day?
The script reports the time it took to apply the log, the size of the log, and the redo apply rate for that log.

Diff(sec) reports the actual time difference between redo logs applied on the standby.
Lag(sec) reports the time difference between when the archive log was completed on the primary and when it was applied on the standby.

If  you come across issues with script below please download it from here

rem Reports standby apply rate with lag

SELECT   TIMESTAMP,
completion_time “ArchTime”,
SEQUENCE#,
ROUND ( (blocks * block_size) / (1024 * 1024), 1) “SizeM”,
ROUND (
(TIMESTAMP
– LAG (TIMESTAMP, 1, TIMESTAMP) OVER (ORDER BY TIMESTAMP))
* 24
* 60
* 60,
1
)
“Diff(sec)”,
ROUND (
(blocks * block_size) / 1024
/ DECODE (
( (TIMESTAMP
– LAG (TIMESTAMP, 1, TIMESTAMP) OVER (ORDER BY TIMESTAMP))
* 24
* 60
* 60),
0,
1,
(TIMESTAMP
– LAG (TIMESTAMP, 1, TIMESTAMP) OVER (ORDER BY TIMESTAMP))
* 24
* 60
* 60
),
1
)
“KB/sec”,
ROUND (
(blocks * block_size) / (1024 * 1024)
/ DECODE (
( (TIMESTAMP
– LAG (TIMESTAMP, 1, TIMESTAMP) OVER (ORDER BY TIMESTAMP))
* 24
* 60
* 60),
0,
1,
(TIMESTAMP
– LAG (TIMESTAMP, 1, TIMESTAMP) OVER (ORDER BY TIMESTAMP))
* 24
* 60
* 60
),
3
)
“MB/sec”,
ROUND (
( (LEAD (TIMESTAMP, 1, TIMESTAMP) OVER (ORDER BY TIMESTAMP))
– completion_time)
* 24
* 60
* 60,
1
)
“Lag(sec)”
FROM   v$archived_log a, v$dataguard_status dgs
WHERE   a.name = REPLACE (dgs.MESSAGE, ‘Media Recovery Log’)
AND dgs.FACILITY = ‘Log Apply Services’
ORDER BY   TIMESTAMP DESC;

Another quickie SQL

@Standby> select snaphot_time,thread#,sequence#,applied_scn from v$standby_apply_snapshot;

If your log application is running slow please alter parallel_execution_message_size parameter to increase the buffer size.

From 10G DataGuard:  Set PARALLEL_EXECUTION_MESSAGE_SIZE = 4096

When using parallel media recovery or parallel standby recovery, increasing the PARALLEL_EXECUTION_MESSAGE_SIZE database parameter to 4K (4096) can improve parallel recovery by as much as 20 percent. Set this parameter on
both the primary and standby databases in preparation for switchover operations. Increasing this parameter requires more memory from the shared pool by each parallel execution slave process.

The PARALLEL_EXECUTION_MESSAGE_SIZE parameter is also used by parallel query operations and should be tested with any parallel query operations to ensure there is sufficient memory on the system. A large number of parallel query
slaves on a 32-bit installation may reach memory limits and prohibit increasing the PARALLEL_EXECUTION_MESSAGE_SIZE from the default 2K


Creating Data Guard Primary/Standby on 2 Windows Servers

Posted by Sagar Patil

Oracle Data Guard is the management, monitoring, and automation software infrastructure that creates, maintains, and monitors one or more standby databases to protect enterprise data from failures, disasters, errors, and corruptions.

The process below is applicable if you have separate Windows/Unix  servers for Datguard primary & Standby Install . For single machine Dataguard Install use my other post.

Oracle Data Guard Overview

Windows 2000 with Oracle 9.2.0.8
Primary Server is called “Primary”
Standby Server is called “Standby”
A working DNS is available for both servers.
Oracle Instance is called DATAGUARD on both Servers
TNS Listener is configured as Primary.world and Standby.world

IMP:
– Configure listener.ora /Tnsnames.ora and add both Primary as well as Standby Database Services
– Always Copy the up -to- date password file from Primary to Standby
– Use Spfile for all configuration settings

Init file for Primary Database InitPrimary.ora
Init file for Standby Database Inittstandby.ora

Sample Listener file listener.ora
Sample Tnsnames file tnsnames.ora

At Primary Database

1. Check Archivelog mode and force database into Archivelog

SQLPLUS>archive log list;
If this returns “Noarchivelog Mode” then
Add following parameters at initPrimary.ora ….
log_archive_dest_1 = ‘location=f:\Oracle\oradata\Primary MANDATORY REOPEN=5’
log_archive_format = “Primary_%S.ARC”
log_archive_start = true
log_buffer = 1000000
log_checkpoint_interval = 10000
log_checkpoint_timeout = 1800

SQL>Startup nomount;
SQL>Alter database archivelog;
Do “alter system switch logfile” and see archive files are dunped at right location

SQL> archive log list;
Standbybase log mode Archive Mode
Automatic archival Enabled
Archive destination f:\Oracle\primary\DATAGUARD
Oldest online log sequence 14
Next log sequence to archive 17
Current log sequence 17

SQL> create spfile from pfile;
File created.

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

At this stage
– COLD backup Primary Database files and ship it to Standby Server.
– At primary create standby control file
SQLPLUS> Alter Database Create Standby Controlfile as ‘D:\Oracle\Oradata\Primary\control.ctl’;
– Copy standby.ctl on to Standby Machine

SQL> startup;
SQL> show parameter spfile;
NAME TYPE VALUE
———————————— ———– ——————————
spfile string %ORACLE_HOME%\StandbyBASE\SPFILE%
ORACLE_SID%.ORA

SQL> connect / as sysdba
Connected.

SQL> alter system set dg_broker_start=TRUE scope=both;
System altered.

C:\Documents and Settings\oracle>dgmgrl
DGMGRL for 32-bit Windows: Version 9.2.0.1.0 – Production.
(c) Copyright 2002 Oracle Corporation. All rights reserved.
Welcome to DGMGRL, type “help” for information.

DGMGRL> connect sys/oracle
Connected.

DGMGRL> show configuration verbose;
Error: ORA-16532: Standby Guard configuration does not exist

SQL> connect / as sysdba
Connected.

SQL> show parameter dg_broker_start
NAME TYPE VALUE
———————————— ———– ——————————
dg_broker_start boolean TRUE

DGMGRL> show configuration verbose;
Error: ORA-16532: Standby Guard configuration does not exist

Above Error indicates we need configuration at Data Guard

Configure Data Guard

Create a new DRC and define a primary Site

DGMGRL> create configuration ‘Primary Site’
as primary site is ‘Primary_site’
resource is ‘Primary_db’
hostname is ‘Primary’
instance name is ‘dataguard’
service name is ‘primary’
site is maintained as physical;
Configuration “Primary Site” added with primary site “Primary_site”
Standbybase resource “Primary_db” added.
Create Secondary Site

DGMGRL> create site ‘Standby_site’
resource is ‘Standby_db’
hostname is ‘Standby’
instance name is ‘dataguard’
service name is ‘standby’
site is maintained as physical;
Site “Standby_site” added to configuration.
Standbybase resource “Standby_db” added.
View DRC config

DGMGRL> show configuration verbose;
Configuration
Name: ‘Primary Site ‘
Enabled: ‘no’
Default state: ‘ONLINE’
Intended state: ‘OFFLINE’
Protection Mode: ‘MaxPerformance’
Number of sites: 2
Sites:
Primary Site: Primary_site
Standby Site: Standby_site
Current status for “Primary Site”:
DISABLED
Enable DRC
Please review alrtlog as well as log for DMON process($BDUMP/alrtDMON.log).

DGMGRL> enable configuration
Enabled.

DGMGRL> show configuration;
Configuration ‘Primary Site’ is
Primary Site is ‘Primary_site’
Standby Site is ‘Standby_site’
Current status for “Genesys Newport”:
SUCCESS

DGMGRL> show resource Primary_db
Resource ‘Primary_db’ on site ‘Primary_site’
depends on ‘Primary_site’
Current status for “Primary_db”:
SUCCESS

DGMGRL> show resource Standby_db
Resource ‘Standby_db’ on site ‘Standby_site’
depends on ‘Standby_site’
Current status for “Standby_db”:
SUCCESS

DGMGRL> exit

At Physical Standby Database

– Configure listener.ora /Tnsnames.ora and add both Primary as well as Standby Database Services
– Always Copy up -to- date password file from Primary to Standby
– Use Spfile for all Configuration Settings
=========================================================================

Add following 2 parameters at standby init.ora file
standby_file_management=auto
standby_archive_dest=’f:\oracle\oradata\standby’

C:\Documents and Settings\oracle>oradim -new -sid DATAGUARD -startmode m

C:\Documents and Settings\oracle>set oracle_sid=DATAGUARD

C:\Documents and Settings\oracle>sqlplus /nolog
SQL*Plus: Release 9.2.0.8.0 – Production on Wed Feb 14 16:12:33 2007
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.

SQL> connect / as sysdba
Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.
Total System Global Area 655843824 bytes
Fixed Size 455152 bytes
Variable Size 335544320 bytes
Database Buffers 318767104 bytes
Redo Buffers 1077248 bytes

SQL> alter database mount standby database;
Database altered.

SQL> recover managed standby database disconnect from session;
Media recovery complete.

SQL> archive log list;
Database log mode Archive Mode
Automatic archival Enabled
Archive destination f:\Oracle\oradata\primary
Oldest online log sequence 14
Next log sequence to archive 17
Current log sequence 17

SQL> create spfile from pfile;
File created.

SQL> shutdown immediate;
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.

SQL> startup mount;
ORACLE instance started.
Total System Global Area 655843824 bytes
Fixed Size 455152 bytes
Variable Size 335544320 bytes
Database Buffers 318767104 bytes
Redo Buffers 1077248 bytes
ORA-01666: controlfile is for a standby database
ORACLE instance shut down.

SQL> startup nomount;
ORACLE instance started.
Total System Global Area 655843824 bytes
Fixed Size 455152 bytes
Variable Size 335544320 bytes
Database Buffers 318767104 bytes
Redo Buffers 1077248 bytes

SQL> alter database mount standby database;
Database altered.

SQL> recover managed standby database disconnect from session;
Media recovery complete.

SQL> alter system set dg_broker_start=TRUE scope=both;
System altered.

SQL> show parameter dg_broker_start;
NAME TYPE VALUE
———————————— ———– ——————————
dg_broker_start boolean TRU

Init file for Primary Database InitPrimary.ora
Init file for Standby Database Inittstandby.ora

Sample Listener file listener.ora
Sample Tnsnames file tnsnames.ora

 

Switchover from primary to standby/ Switchover Primary : No Dgmgrl

Posted by Sagar Patil

This process will reverse database roles in a Data Guard setup, i.e. standby database becomes primary.

It’s assumed that:

  1. You already have working Data Guard setup between primary and standby
  2. Archive redo is begin shipped and applied from primary to standby
  3. Archive redo application is up to date between primary & standby

Requirements :

  1. All applications must be shutdown
  2. Switchover must be initiated on primary database before being completed on standby database
  3. The primary database must be open, the standby database must be mounted and in managed recovery mode

Note
When you perform a switchover the controlfile type is converted in-place, i.e. the primary controlfile becomes a standby controlfile and vice-versa. You must be CAREFUL which database you are working on…..

On the STANDBY database

1. Check archive_lag_target

SQL> show parameters lag

NAME TYPE VALUE ———————————— ———– —————————-
archive_lag_target integer 0

If non-zero set to zero using

SQL> alter system set archive_lag_target=0 scope=both;

On the PRIMARY database

1. Shutdown RAC databases

(RAC environment only) Shutdown all but one of the primary database instances.

2. Verify it is possible to switchover

SQL> SELECT SWITCHOVER_STATUS FROM V$DATABASE;

SWITCHOVER_STATUS

——————

SESSIONS ACTIVE

Switchover requires that there be only one active session (the one performing the switchover).

Documentation states the above query should return “TO STANDBY” for switchover to succeed but it – always returns “SESSIONS ACTIVE” because of the one active session – Problemo!!

The key point is to make sure there is only one user connected – SYS – check V$SESSION

3. Initiate the switchover

SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY WITH SESSION SHUTDOWN;

Database altered.

This converts the primary database to standby. Current controlfile is backed up to trace should you need to reverse the switchover. At this stage we temporarily have two standby databases…

4. Shutdown, restart and mount as standby

SQL> SHUTDOWN NORMAL;

ORA-01507: database not mounted <= This can be ignored

ORACLE instance shut down.

SQL> STARTUP NOMOUNT;
ORACLE instance started.
Total System Global Area 920912008 bytes
Fixed Size 711896 bytes
Variable Size 285212672 bytes
Database Buffers 111554492 bytes
Redo Buffers 811008 bytes

SQL> ALTER DATABASE MOUNT STANDBY DATABASE;

Database altered.

5. Verify switchover status

SQL> SELECT SWITCHOVER_STATUS FROM V$DATABASE;

SWITCHOVER_STATUS

——————

SESSIONS ACTIVE

According to documentation this should return “SWITCHOVER PENDING”, same catch as before, there has to be one session active to do the switchover.

6. Verify switchover status

SQL> SELECT SWITCHOVER_STATUS FROM V$DATABASE;

SWITCHOVER_STATUS

——————

SESSIONS ACTIVE

According to the documentation this should return “SWITCHOVER PENDING” but I have only ever seen “SESSIONS ACTIVE”, same situation as before – there has to be one session active to do the switchover.

On the STANDBY database

6. Switch standby database to primary

SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY ;

Database altered.

7. Shutdown and restart the new primary database

SQL> SHUTDOWN

ORA-01507: database not mounted <= This can be ignored

ORACLE instance shut down.

SQL> STARTUP
ORACLE instance started.
Total System Global Area 920912008 bytes
Fixed Size 711896 bytes
Variable Size 285212672 bytes
Database Buffers 111554492 bytes

Redo Buffers 811008 bytes
Database mounted.
Database opened.

The database is now the primary database.

On the STANDBY database (the original primary)

8. Start managed recovery and log apply services and set archive lag target

SQL> ALTER SYSTEM SET ARCHIVE_LAG_TARGET=900 SCOPE=BOTH;

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;

Database altered.

On the PRIMARY database (the original standby)

9. Begin archiving logs

SQL> ALTER SYSTEM ARCHIVE LOG START;

System altered.

SQL> ALTER SYSTEM SWITCH LOGFILE;

System altered.

On BOTH databases

10. Check the alert logs to make sure archived redo is being sent and received

The primary alert log should look like this

ARC1: Evaluating archive log 3 thread 1 sequence 30

ARC1: Archive destination LOG_ARCHIVE_DEST_2: Previously completed

ARC1: Beginning to archive log 3 thread 1 sequence 30

Creating archive destination LOG_ARCHIVE_DEST_1: ‘/u01/app/oracle/archive/arch/DATAGD_1_30.arc’

ARC1: Completed archiving log 3 thread 1 sequence 30

The standby alert log should look like this

Media Recovery Waiting for thread 1 seq# 30 (in transit)

Fri Sep 3 10:12:14 2004

Media Recovery Log  ‘/u01/app/oracle/archive/arch/DATAGD_1_30.arc

Top of Page

Top menu