Since your standby was lagging by 300+ archive logs, the purpose of this activity was to resynchronize the standby database quickly using incremental recovery from the primary database service instead of waiting for MRP to apply hundreds of archived logs sequentially.
Scenario
Environment:
Primary : ORCL
Standby : ORCLDG
RAC : 2 Nodes
Database : Oracle 19c
Show more lines
Problem:
Standby was significantly behind the Primary
300+ archive logs pending
Redo Apply lag increasing
Show more lines
Typical reasons:
• Network interruption
• Data Guard transport issue
• Restore point creation/deletion
• Disk space issue
• MRP stopped for long duration
• Archive gap accumulation
Instead of applying 300+ archives manually, RMAN was used to synchronize the standby directly from the primary database.
________________________________________
Step 1: Change RMAN Configuration
Executed on both Primary and Standby:
SQL
CONFIGURE DEFAULT DEVICE TYPE TO DISK;
CONFIGURE CHANNEL DEVICE TYPE SBT CLEAR;
CONFIGURE DEVICE TYPE DISK PARALLELISM 8 BACKUP TYPE TO BACKUPSET;
CONFIGURE DEVICE TYPE 'SBT_TAPE' CLEAR;
Show more lines
Why?
Your RMAN configuration was still pointing to:
SBT_TAPE
Show more lines
which means RMAN was expecting a media manager such as:
NetBackup
TSM
Networker
Commvault
Show more lines
When RECOVER STANDBY DATABASE FROM SERVICE runs, Oracle internally creates backup sets and transfers changed blocks through the network.
We wanted RMAN to use:
DISK only
Show more lines
therefore:
SBT channels removed
Default device set to DISK
Show more lines
This prevents RMAN from trying to allocate tape channels.
________________________________________
Step 2: Check Standby Structure
SQL
RMAN> REPORT SCHEMA;
Show more lines
Why?
To verify:
Datafile numbers
Tablespaces
File names
ASM locations
Show more lines
and ensure Primary and Standby structures match.
Important before any recovery activity.
________________________________________
Step 3: Mount Only One Standby Instance
Stopped Standby:
Shell
srvctl stop database -d ORCLDG
Show more lines
Verified:
Shell
srvctl status database -d ORCLDG
`
Show more lines
Output:
ORCLDG1 not running
ORCLDG2 not running
Show more lines
Started only one instance:
Shell
srvctl start instance -d ORCLDG -i ORCLDG1 -o mount
Show more lines
________________________________________
Why only one instance?
RECOVER STANDBY DATABASE FROM SERVICE
requires:
Standby mounted
No active redo apply
Single recovery instance
Show more lines
Running multiple RAC instances may cause:
File locking
Recovery conflicts
Controlfile enqueue contention
Show more lines
Therefore:
One instance in MOUNT state
Show more lines
is Oracle best practice.
________________________________________
Why MOUNT Mode?
Checked:
SQL
show pdbs;
Show more lines
Output:
PDB$SEED MOUNTED
PDB01 MOUNTED
Show more lines
When database is mounted:
Controlfile available
Datafiles accessible
No user activity
Recovery possible
Show more lines
RMAN can safely update datafiles.
________________________________________
Step 4: Recover Standby from Primary
Executed:
RECOVER STANDBY DATABASE FROM SERVICE ORCL;
Show more lines
________________________________________
What does Oracle do internally?
Oracle connects to:
Primary Service = ORCL
Show more lines
and performs:
Phase 1
Compare SCNs
Primary SCN
Standby SCN
Show more lines
Determine required changes.
________________________________________
Phase 2
Create Incremental Backup
On Primary:
Changed blocks identified
Show more lines
RMAN creates incremental backup sets.
________________________________________
Phase 3
Network Transfer
Instead of copying files:
Primary --> Standby
Show more lines
changes are streamed over Oracle Net.
________________________________________
Phase 4
Recovery
Standby datafiles updated.
Result:
Standby catches up quickly
Show more lines
instead of applying 300+ archives.
________________________________________
Step 5: Failure
You received:
ORA-19554: error allocating device, device type: SBT_TAPE
ORA-27211: Failed to load Media Management Library
Show more lines
________________________________________
Why did it happen?
Although:
SQL
CONFIGURE CHANNEL DEVICE TYPE SBT CLEAR;
Show more lines
was executed,
RMAN still had:
SQL
CONFIGURE DEVICE TYPE 'SBT_TAPE'
Show more lines
configured.
Oracle therefore attempted to allocate:
SBT_TAPE channel
Show more lines
and looked for media management libraries.
Since the library wasn't available:
ORA-27211
Show more lines
occurred.
________________________________________
Fix
Executed:
SQL
CONFIGURE DEVICE TYPE 'SBT_TAPE' CLEAR;
Show more lines
This removes tape configuration completely.
________________________________________
Step 6: Recovery Successful
Executed again:
SQL
rman isn’t fully supported. Syntax highlighting is based on SQL.
RECOVER STANDBY DATABASE FROM SERVICE ORCL;
`
Show more lines
Output:
media recovery complete
Show more lines
________________________________________
What happened?
Oracle:
Connected to Primary
Generated incremental changes
Transferred blocks
Updated standby files
Applied required redo
Show more lines
Standby SCN became nearly equal to Primary SCN.
Therefore:
media recovery complete
Show more lines
was displayed.
Elapsed:
00:00:12
Show more lines
which shows RMAN synchronized using block recovery instead of processing hundreds of archives manually.
________________________________________
Step 7: Restart Managed Recovery Process (MRP)
Executed:
SQL
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;
Show more lines
________________________________________
Why?
RECOVER STANDBY DATABASE FROM SERVICE
only synchronizes datafiles.
After completion:
MRP is not automatically started.
Show more lines
Need to restart redo apply.
Command:
SQL
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;
Show more lines
starts background recovery.
________________________________________
Verify MRP
Output:
MRP0 APPLYING_LOG
Show more lines
Example:
MRP0 APPLYING_LOG 2 22063
Show more lines
This is the most important line.
Meaning:
MRP started successfully
Standby is applying incoming redo
Data Guard healthy
``
Show more lines
________________________________________
Understanding Other Processes
ARCH
ARCH CLOSING
Show more lines
Means:
Archive log received
Archive log being finalized
``
Show more lines
________________________________________
ARCH CONNECTED
ARCH CONNECTED
Show more lines
Means:
RFS/ARCH waiting for new redo
Show more lines
Normal status.
________________________________________
DGRD
DGRD ALLOCATED
Show more lines
Data Guard Broker processes.
Normal condition.
________________________________________
MRP0
MRP0 APPLYING_LOG
Show more lines
Most critical process.
Means:
Redo Apply is active.
Show more lines
________________________________________
Overall Theory
When a standby falls behind by a large number of archives (300+ in your case), applying archived logs one by one can take a long time.
Oracle provides:
rman isn’t fully supported. Syntax highlighting is based on .
RECOVER STANDBY DATABASE FROM SERVICE <Primary_Service>;
Show more lines
which:
1. Connects to Primary.
2. Compares SCNs.
3. Creates incremental backups of changed blocks.
4. Transfers them over the network.
5. Updates standby datafiles.
6. Brings standby very close to Primary.
7. MRP is then restarted to continue normal redo apply.
This approach is significantly faster than manually shipping and applying hundreds of archive logs and is the recommended method when the standby is heavily lagging.
No comments:
Post a Comment