Disclaimer

Thursday 7 December 2023

Physical Standby Data Guard useful SQL scripts

Physical Standby Data Guard useful SQL scripts

Below are some of the frequently used Physical Standby Data Guard related SQL queries

1) Basic information of database (primary or standby)

SQL> SELECT DATABASE_ROLE, DB_UNIQUE_NAME INSTANCE, OPEN_MODE, PROTECTION_MODE, PROTECTION_LEVEL, SWITCHOVER_STATUS FROM V$DATABASE;

DATABASE_ROLE INSTANCE OPEN_MODE PROTECTION_MODE PROTECTION_LEVEL SWITCHOVER_STATUS

–––––––– ––––––––––––––– ––––––––––

PHYSICAL STANDBY stdby READ ONLY MAXIMUM PERFORMANCE MAXIMUM PERFORMANCE NOT ALLOWED

2) Check for messages/errors

SQL> SELECT MESSAGE FROM V$DATAGUARD_STATUS;

MESSAGE

––––––––––––––––––––––––––––––––––––––––

ARC0: Archival started

ARC1: Archival started

ARC2: Archival started

ARC2: Becoming the ‘no FAL’ ARCH

ARC1: Becoming the heartbeat ARCH

ARC1: Becoming the active heartbeat ARCH

ARC3: Archival started

3) To display current status information for specific physical standby database background processes.

SQL> SELECT PROCESS, STATUS, THREAD#, SEQUENCE#, BLOCK#, BLOCKS FROM V$MANAGED_STANDBY ;

PROCESS STATUS THREAD# SEQUENCE# BLOCK# BLOCKS

––––- –––––– ––––– ––––– ––––– –––––

ARCH CONNECTED 0 0 0 0

ARCH CONNECTED 0 0 0 0

ARCH CLOSING 1 54 45056 755

ARCH CLOSING 1 57 1 373

RFS IDLE 0 0 0 0

RFS IDLE 0 0 0 0

RFS IDLE 0 0 0 0

RFS IDLE 1 58 30239 1

 

8 rows selected.

4) Show received archived logs on physical standby

Run this query on physical standby

SQL> select registrar, creator, thread#, sequence#, first_change#, next_change# from v$archived_log;

REGISTR CREATOR THREAD# SEQUENCE# FIRST_CHANGE# NEXT_CHANGE#

–––- –––- ––––– ––––– ––––––- ––––––

RFS ARCH 1 29 1630326 1631783

RFS ARCH 1 30 1631783 1632626

RFS LGWR 1 31 1632626 1669359

RFS ARCH 1 33 1676050 1676124

RFS ARCH 1 32 1669359 1676050

RFS ARCH 1 35 1681145 1681617

RFS ARCH 1 34 1676124 1681145

RFS ARCH 1 37 1688494 1688503

RFS ARCH 1 36 1681617 1688494

RFS ARCH 1 38 1688503 1689533

RFS LGWR 1 39 1689533 1697243

5) To check the log status

SQL> select ‘Last Log applied : ‘ Logs, to_char(next_time,‘DD-MON-YY:HH24:MI:SS’) Time

from v$archived_log

where sequence# = (select max(sequence#) from v$archived_log where applied=’YES’)

union

select ‘Last Log received : ‘ Logs, to_char(next_time,‘DD-MON-YY:HH24:MI:SS’) Time

from v$archived_log

where sequence# = (select max(sequence#) from v$archived_log);

LOGS TIME

–––––––––– –––––––––––––-

Last Log applied : 24-MAR-14:10:11:10

Last Log received : 27-MAR-14:12:40:17

6) To display various information about the redo data. This includes redo data generated by the primary database that is not yet available on the standby database and how much redo has not yet been applied to the standby database.

set lines 132

col value format a20

SQL> select name, value from V$DATAGUARD_STATS;

NAME VALUE

––––––––––––––– ––––––––––

transport lag +00 00:00:00

apply lag

apply finish time

estimated startup time 23

7) to monitor efficient recovery operations as well as to estimate the time required to complete the current operation in progress:

SQL> select to_char(start_time, ‘DD-MON-RR HH24:MI:SS’) start_time,

item, round(sofar/1024,2) “MB/Sec”

from v$recovery_progress

where (item=‘Active Apply Rate’ or item=‘Average Apply Rate’);

START_TIME ITEM MB/SEC

–––––- –––––––––––––––- ––––

27-MAR-14 15:49:44 Active Apply Rate 8.5

27-MAR-14 15:49:44 Average Apply Rate 6.30

8) To find last applied log

SQL> select to_char(max(FIRST_TIME),‘hh24:mi:ss dd/mm/yyyy’) FROM V$ARCHIVED_LOG where applied=‘YES’;

TO_CHAR(MAX(FIRST_T

–––––––––-

10:11:08 24/03/2014

9) To see if standby redo logs have been created. The standby redo logs should be the same size as the online redo logs. There should be (( # of online logs per thread + 1) * # of threads) standby redo logs. A value of 0 for the thread# means the log has never been allocated.

SQL> SELECT thread#, group#, sequence#, bytes, archived, status FROM v$standby_log order by thread#, group#;

THREAD# GROUP# SEQUENCE# BYTES ARC STATUS

––––– ––––– ––––– ––––– –- –––––––––-

1 8 0 104857600 NO UNASSIGNED

1 9 58 104857600 YES ACTIVE

1 10 0 104857600 NO UNASSIGNED

1 11 0 104857600 YES UNASSIGNED

10) To produce a list of defined archive destinations. It shows if they are enabled, what process is servicing that destination, if the destination is local or remote, and if remote what the current mount ID is. For a physical standby we should have at least one remote destination that points the primary set.

column destination format a35 wrap

column process format a7

column ID format 99

column mid format 99

SQL> SELECT thread#, dest_id, destination, gvad.status, target, schedule, process, mountid mid FROM gv$archive_dest gvad, gv$instance gvi WHERE gvad.inst_id = gvi.inst_id AND destination is NOT NULL ORDER BY thread#, dest_id;

THREAD# DEST_ID DESTINATION STATUS TARGET SCHEDULE PROCESS MID

–––– –––- –––––––––––––––––––––––––––––––––––––––––

1 1 USE_DB_RECOVERY_FILE_DEST VALID LOCAL ACTIVE ARCH 0

1 2 brij VALID REMOTE PENDING LGWR 0

1 32 USE_DB_RECOVERY_FILE_DEST VALID LOCAL ACTIVE RFS 0

11) Verify the last sequence# received and the last sequence# applied to standby database.

SQL> SELECT al.thrd “Thread”, almax “Last Seq Received”, lhmax “Last Seq Applied” FROM (select thread# thrd, MAX(sequence#) almax FROM v$archived_log WHERE resetlogs_change#=(SELECT resetlogs_change# FROM v$database) GROUP BY thread#) al, (SELECT thread# thrd, MAX(sequence#) lhmax FROM v$log_history WHERE resetlogs_change#=(SELECT resetlogs_change# FROM v$database) GROUP BY thread#) lh WHERE al.thrd = lh.thrd;

Thread Last Seq Received Last Seq Applied

–––––    ––––––––-           ––––––––

1         57                53

 

Determining Dataguard Standby Apply Rate

SQL> select snapshot_time, thread#, sequence#, applied_scn, applied_time, apply_rate
from V$standby_apply_snapshot;

SNAPSHOT_TIME     THREAD# SEQUENCE# APPLIED_SCN APPLIED_TIME  APPLY_RATE
----------------- ------- --------- ----------- ------------- ----------
27-02-23 15:45:08     1      8684    2549146933 27-02-08 15:44:48  47234
27-02-23 15:45:08     2      7145    2549146847 27-02-08 15:44:49  47234

 

You can see from the applied_time how far behind you are in applying. So is my apply rate 47MB per second


Wednesday 29 November 2023

TX-Row Lock contention and Large execution time/high IO

 Observations


1. Heavy TX-Row Lock contention issues has been observed in the system .

2. Large execution time/high io has been experienced for few queries .


Reasons & Facts


1. TX-Row lock contention happens when multiple users tries to manipulate the record which is hold by another 

user for modification .


Fact : Upon investigation we were able to capture below scenario where user A130796 and A124826 are 

waiting for the same record.




2. Large execution time has been experienced for the below query


Sql Statement


SELECT ROWID,PROC_STAT,SEQ_NO,PAN,SEC,ATT_YR,DT_FILED

FROM SS_RECT_APL

WHERE ATT_YR=:1 AND PROC_STAT=:2

ORDER BY PAN,ATT_YR


Execution Statistics


PLAN_TABLE_OUTPUT


----------------------------------------------------------------------------------------------------------------------------

Plan hash value: 3781133068

-------------------------------------------------------------------------------------------------------

| Id | Operation       | Name                             | Rows | Bytes   | Cost (%CPU)| Time |

-------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT |                                  | 7233 | 339K    | 37920 (2)| 00:00:46 |

| 1 | SORT ORDER BY    |                                  | 7233 | 339K    | 37920 (2)| 00:00:46 |

|* 2| TABLE ACCESS BY INDEX ROWID| T_RECT_APL             | 7233 | 339K    | 37916 (2)| 00:00:46 |

|* 3| INDEX RANGE SCAN | INDX_T_RECT_APL_ATT_YR           |          366K| | 1257 (5)| 00:00:02 |

-------------------------------------------------------------------------------------------------------


Predicate Information (identified by operation id):

---------------------------------------------------

2 - filter("PROC_STAT"=:2)

3 - access("ATT_YR"=TO_NUMBER(:1))



Fact : Upon investigation it has been noticed that above query is taking too much I/O 37920 and hence large execution time for the same .


Recommendations


1. For resolving TX-row lock issues make sure that no two users were working on the same record.


2. For increasing the efficiency and decreasing the execution time of the query listed above 

  a composite index need to be made on ATT_YR,PROC_STAT columns of table ATT.T_RECT_APL .



Tuesday 17 October 2023

VALIDATE STATIC CONNECT IDENTIFIER failed for Primary Database - Dataguard

 I have been encountered below issue while validating static connect identifier.


ORA-12520: TNS:listener could not find available handler for requested type of server


DGMGRL> VALIDATE STATIC CONNECT IDENTIFIER FOR all;

Oracle Clusterware is not configured on database "ORCL11".

Connecting to database "ORCL11" using static connect identifier "(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=198.164.1.74)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORCL11)(INSTANCE_NAME=ORCL11)(SERVER=DEDICATED)(STATIC_SERVICE=TRUE)))" ...

Unable to connect to database using (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=198.164.1.74)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORCL11)(INSTANCE_NAME=ORCL11)(SERVER=DEDICATED)(STATIC_SERVICE=TRUE)))

ORA-12520: TNS:listener could not find available handler for requested type of server

Failed.

Oracle Clusterware is not configured on database "ORCL11db".

Connecting to database "ORCL11db" using static connect identifier "(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=ORCL11DBA01)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORCL11DB)(INSTANCE_NAME=ORCL11DB)(SERVER=DEDICATED)(STATIC_SERVICE=TRUE)))" ...

Succeeded.

The static connect identifier allows for a connection to database "ORCL11db".


Checked : Below was my listener.ora file which was Dynamic that's why I was getting error  


$ cat listener.ora
LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = ORCL11DBA01)(PORT = 1521))
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
    )
  )

SUBSCRIBE_FOR_NODE_DOWN_EVENT_LISTENER=OFF


Solution :- 

I have added entry and made the statis listener 

$ cat listener.ora

LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = ORCL11DBA01)(PORT = 1521))
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
    )
  )

SID_LIST_LISTENER=
  (SID_LIST =
    (SID_DESC =
      (GLOBAL_DBNAME = ORCL11DB)
      (ORACLE_HOME = /applications/oracle/19.3.0.0)
      (SID_NAME =ORCL11DB)
    )
)

SUBSCRIBE_FOR_NODE_DOWN_EVENT_LISTENER=OFF



After this ORA-12520: TNS:listener could not find available handler for requested type of server issue resolved 



DGMGRL> VALIDATE STATIC CONNECT IDENTIFIER FOR all;

Oracle Clusterware is not configured on database "ORCL11".

Connecting to database "ORCL11" using static connect identifier "(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=198.164.1.74)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORCL11)(INSTANCE_NAME=ORCL11)(SERVER=DEDICATED)(STATIC_SERVICE=TRUE)))" ...

Succeeded.

The static connect identifier allows for a connection to database "ORCL11".


Oracle Clusterware is not configured on database "ORCL11db".

Connecting to database "ORCL11db" using static connect identifier "(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=ORCL11DBA01)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORCL11DB)(INSTANCE_NAME=ORCL11DB)(SERVER=DEDICATED)(STATIC_SERVICE=TRUE)))" ...

Succeeded.

The static connect identifier allows for a connection to database "ORCL11db".



Now I can do switchover and switchback using dgmgrl utility :)

Happy Learning ...!!!



Thursday 5 October 2023

Database performance Note-1

 High resource intensive SQL queries.

**************************************

SQL_ID : 4ff82tr5w35f2 

SQL profiling is done with best available plan in OLTP production Database . 

Please find below current execution details . We will keep a eye on this SQL .


SQL_ID        MODULE     PLAN_HASH_VALUE CHILD_NUMBER EXECUTIONS    Bg/gets   Ela/Exec DT

------------- ---------- --------------- ------------ ---------- ---------- ---------- ------------

4ff82tr5w35f2 siebmtshmw      3743305940            1        147  125.85034 .002336075 02-SEP 15:25


SQL_ID : an5zgpp56sqd9 &cjdakywq1grpa

execution Plan of the SQl is already optimised , Execution of the SQL is high . 

So if any further filters can be applied which can further reduce dataset it is retuning can help . 


High Disk File IO Tablespace

*****************************

1. SIEBEL_DATA & SIEBE_INDEX  are kept at slow speed disc ( decision taken by management) also 

the IO ( read and write on these tablespaces is minimal 

( hardly 3-4 users have access to this data at any day) .

This decision is to be taken by ASG management if Archived data needs to be moved to High speed disk .


2. We can plan to move indexes from SIEBEL_DATA to SIEBEL_INDEX.


3. Stats Gather was done just after completion of archiving cycle . After that no data is changed in tables of these schema . 

  I would request Oracle to suggest if we need to keep gathering stats for static tables as well on weekly basis as best practices.

  

High Fragmentation

*****************************

In OLTP production database we have multiple time tried De fragmentation of tables ( along with ACS team - Santosh kumar) 

using traditional defragmentaion method ( Alter table <> move ) and also using Online Reorg . 

But in both cases we endup having bigger table size then before activity (ie. fragmentation further increases).

We have already raised a SR with oracle on this.

Please suggest if there is any other approach which can be followed to achieve desired results .



ITL Waits

**************

We can implement the change. But as users will keep increasing , concurrency will also increase hence we have to prepare 

a benchmarking till what value we can increase INITRANS for listed indexes in recommendation from Oracle.



Stale Gather Statistics

*****************************

We have stats gather policy implemented for All siebel tables( application) - Weekly Stats gather every sunday Night .

Please suggest if we need to gather stats for all objects/Tables in database on weekly basis even if these are static table. 

If yes then we have to get new Stats gather script generated from Oracle End to be implemented in Production.



SGA Analysis

*****************************

We can go ahead and implement changes suggested . As we are just increasing values of different component without 

increasing total memory allotted for OLTP instance .


Thursday 28 September 2023

ASM errors ORA-15032 and ORA-15041 while adding disk

We have faced below ASM errors while adding disks and we have 2 node RAC database with NORMAL redundancy at ASM level.

We were trying to add 500 GB ASM disk on ASM instance and at the end of ASM operation we got below error.


DBT-30007 Addition of disks to disk group DATA01 failed.

ORA-15032: not all alterations performed

ORA-15041: diskgroup "DATA01" space exhausted






SOLUTION

-- To implement the solution, please execute the following steps::


1) Review diskgroup configuration, check how many disks within each failure group.

2) Add the remaining disks appropriately so that each failgroup has equally numbered disks.


We can see from ASM alert log below

2023-08-17T05:21:10.203829+02:00
NOTE: Disk _DROPPED_0008_DATA01 in mode 0x0 marked for de-assignment
SUCCESS: refreshed membership for 2/0xe06c1f5c (DATA01)
2023-08-17T05:21:11.997291+02:00
NOTE: Attempting voting file refresh on diskgroup DATA01
NOTE: Refresh completed on diskgroup DATA01. No voting file found.
2023-08-17T05:22:19.072663+02:00
WARNING: Negative disk usage {0, -42698) found for disk 8
WARNING: Negative used space found in at least one disk for grp DATA01 (2/0xe06c1f5c)


V$ASM_DISK View Shows Some Disk Header Status as IGNORED and Group Number as "0". ( Doc ID 1299866.1 )




Run the below command for adding disk 


SQL> alter diskgroup DATA add failgroup FG1 disk '/dev/oracleasm/disks/DATA09' force ;


After that we could successfully see that issue was resolved .


=================================================

Adding disk in ASM :-


SQL> alter diskgroup DATA01 rebalance power 10;



SQL> alter diskgroup DATA01 add
failgroup FG1 disk '/dev/oracleasm/disks/DATA43' name DATA43
failgroup FG2 disk '/dev/oracleasm/disks/DATA44' name DATA44;








100 Oracle DBA Interview Questions and Answers

  Here are 100 tricky interview questions tailored for a Senior Oracle DBA role. These questions span a wide range of topics, including perf...