RAC : Apply PSU Patch To Standard Cluster 12c
Oracle RAC 12.2.0.1 Patch 30920127(Apr 2020)
Patch 30920127: Oracle Database - Enterprise Edition 12.2.0.1.0 for Linux x86-64 - GI APR 2020 RELEASE UPDATE 12.2.0.1.200414
Mentioned steps to apply patch for GI and RDBMS homes and database using rolling method with minimum to no downtime can be used for any quarterly patches, however this particular article was prepared while applying Jan'20 PSU patch -Oracle Database Patch 30920127 - GI Jan 2020 Release Update 12.2.0.1.200414 on LINUS 64 bit systems.
Below mentioned steps will give you fair idea to apply PSU Patch, however I would highly encourage you to go through the patch readme thoroughly before proceeding with the patching. It's always better to apply patch on lower environment first before applying it on production environment.
First of all identify the patches required to be applied for your environment and download and keep zip files in specific location on the server. for example I have downloaded and kept patches zip files at below location on my all nodes of the RAC cluster.
[oracle@rac3 30920127]$
[oracle@rac3 30920127]$ pwd
/backup/30920127
[oracle@rac3 30920127]$
[oracle@rac3 30920127]$ ls -lrt
total 152
-rwxrwxrwx 1 root root 0 Apr 1 2020 README.txt
drwxrwxrwx 2 root root 4096 Apr 1 2020 automation
drwxrwxrwx 5 root root 4096 Apr 1 2020 30869447
drwxrwxrwx 4 root root 4096 Apr 1 2020 26839277
drwxrwxrwx 5 root root 4096 Apr 1 2020 30882603
drwxrwxrwx 5 root root 4096 Apr 1 2020 30886680
drwxrwxrwx 4 root root 4096 Apr 1 2020 30888810
-rwxrwxrwx 1 root root 121082 Apr 1 2020 README.html
-rwxrwxrwx 1 root root 5828 Apr 1 2020 bundle.xml
Few Important points:
1. This document can be used to apply PSU patch for Unix/Linux and some of the portion can be used for Windows as well.
2. This method is used to apply patch for 12cR2, which is quite different than application of patch in 11g and prior.
3. Binary Patching activity will be performed as a root user.
4. Only GI patch needs to be downloaded for RAC(which includes GI+RDBMS components), same patch needs to be applied on both Grid Infra home and RDBMS Home.
5. datapatch, which is post SQL Load/execution needs to performed only once on any single node , when binary patching completed on all/both the nodes.
6. It's recommended that patch related readme needs to be followed , there may be small changes here and there considering the patch version and OS version.
7. For RAC environment, Rolling forward method is the best as there will NOT be complete application downtime, however in flight transaction may impact while performing activity.
1. This document can be used to apply PSU patch for Unix/Linux and some of the portion can be used for Windows as well.
2. This method is used to apply patch for 12cR2, which is quite different than application of patch in 11g and prior.
3. Binary Patching activity will be performed as a root user.
4. Only GI patch needs to be downloaded for RAC(which includes GI+RDBMS components), same patch needs to be applied on both Grid Infra home and RDBMS Home.
5. datapatch, which is post SQL Load/execution needs to performed only once on any single node , when binary patching completed on all/both the nodes.
6. It's recommended that patch related readme needs to be followed , there may be small changes here and there considering the patch version and OS version.
7. For RAC environment, Rolling forward method is the best as there will NOT be complete application downtime, however in flight transaction may impact while performing activity.
Environment Details:-
1) Patch Installation:-
==>1.2.1 Patch Installation Prerequisites
Note: Make sure that crontab jobs or any other scheduled jobs should be commented during patching.
-->1.2.1.1 OPatch Utility Information
You must use the OPatch utility version 12.2.0.1.17 or later to apply this patch for all platforms. Oracle recommends that you use the latest released OPatch version for 12.1 releases,
which is available for download from My Oracle Support patch 6880880 by selecting ARU link for the 12.2.0.1.0 release. It is recommended that you download the Opatch utility and
the patch in a shared location to be able to access them from any node in the cluster for the patch application on each node.
When patching the GI Home, a shared location on ACFS only needs to be unmounted on the node where the GI Home is being patched.
The new opatch utility should be updated in all the Oracle RAC database homes and the GI home that are being patched.
Note: Make sure that crontab jobs or any other scheduled jobs should be commented during patching.
-->1.2.1.1 OPatch Utility Information
You must use the OPatch utility version 12.2.0.1.17 or later to apply this patch for all platforms. Oracle recommends that you use the latest released OPatch version for 12.1 releases,
which is available for download from My Oracle Support patch 6880880 by selecting ARU link for the 12.2.0.1.0 release. It is recommended that you download the Opatch utility and
the patch in a shared location to be able to access them from any node in the cluster for the patch application on each node.
When patching the GI Home, a shared location on ACFS only needs to be unmounted on the node where the GI Home is being patched.
The new opatch utility should be updated in all the Oracle RAC database homes and the GI home that are being patched.
[grid@rac3 ~]$ cd $ORACLE_HOME/OPatch
[grid@rac3 OPatch]$
[grid@rac3 OPatch]$
[grid@rac3 OPatch]$ ./opatch version
OPatch Version: 12.2.0.1.19
OPatch succeeded.
[oracle@rac3 ~]$
[oracle@rac3 ~]$ cd $ORACLE_HOME/OPatch
[oracle@rac3 OPatch]$
[oracle@rac3 OPatch]$
[oracle@rac3 OPatch]$ ./opatch version
OPatch Version: 12.2.0.1.19
OPatch succeeded.
[grid@rac4 ~]$ cd $ORACLE_HOME/OPatch
[grid@rac4 OPatch]$
[grid@rac4 OPatch]$ ./opatch version
OPatch Version: 12.2.0.1.19
OPatch succeeded.
[oracle@rac4 ~]$ cd $ORACLE_HOME/OPatch
[oracle@rac4 OPatch]$
[oracle@rac4 OPatch]$
[oracle@rac4 OPatch]$ ./opatch version
OPatch Version: 12.2.0.1.19
OPatch succeeded.
Download the OPatch utility to a temporary directory.
For each Oracle RAC database home and the GI home that are being patched, run the commands as the home owner to extract the OPatch utility.
In our case we have OPatch Version: 12.2.0.1.19 for GRID and ORACLE user.
2) Backup :-
For GI Home: Run it as GRID user
-->To run backup of home in background
cd /backup/PATCH/Jan2020
nohup tar cvf - /u01/app/12.2.0/grid | gzip > /backup/PATCH/Jan2020/GRID_HOME_bkp1_30012020.tar.gz &
-->to run backup of home in foreground
tar cvf - /u01/app/12.2.0/grid | gzip > /backup/PATCH/Jan2020/Grid_home_bkp1_30012020.tar.gz
OH home: RUN it as Oracle user
-->To run backup of home in background
cd /backup/PATCH/Jan2020
nohup tar cvf - /u01/app/oracle/product/12.2.0.1/dbhome_1 | gzip > ORACLE_HOME_bkp1_30012020.tar.gz &
OR
-->to run backup of home in foreground
cd /backup/PATCH/Jan2020
tar cvf - /u01/app/oracle/product/12.2.0.1/dbhome_1 | gzip > /backup/PATCH/Jan2020/ORACLE_HOME_bkp1_08102019.tar.gz
3) Validation of Oracle Inventory:-
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
Note:Perform this on both the nodes:
For Grid Home:
$ /u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_B4.lst
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_detail_B4.lst
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
Note:Perform this on both the nodes:
For Grid Home:
$ /u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_B4.lst
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_detail_B4.lst
For RDBMS Home as oracle user:
If this command succeeds, it lists the Oracle components that are installed in the home. Save the output so you have the status prior to the patch apply.
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_B4.lst
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_detail_B4.lst
If this command succeeds, it lists the Oracle components that are installed in the home. Save the output so you have the status prior to the patch apply.
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_B4.lst
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_detail_B4.lst
~~>>Execute below SQL to check the status of the patches from DB:
sqlplus / as sysdba
spool /backup/PATCH/Jan2020/datapatch_status_B4.lst
set line 500
col patch_id for 9999999999
col patch_uid for 9999999999
col version for a10
col action for a10
col install_id for 99
col action_time for a30
col description for a70
col bundle_series for a20
select patch_id, patch_uid, version, action, install_id, action_time, description, bundle_series from dba_registry_sqlpatch;
spool off
sqlplus / as sysdba
spool /backup/PATCH/Jan2020/datapatch_status_B4.lst
set line 500
col patch_id for 9999999999
col patch_uid for 9999999999
col version for a10
col action for a10
col install_id for 99
col action_time for a30
col description for a70
col bundle_series for a20
select patch_id, patch_uid, version, action, install_id, action_time, description, bundle_series from dba_registry_sqlpatch;
spool off
~~>1.2.1.2 Validation of Oracle Inventory
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
$ <ORACLE_HOME>/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
~~>1.2.1.3 Download and Unzip the Patch : I assume this step has been done
4) Run OPatch Conflict Check:-
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
$ <ORACLE_HOME>/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
~~>1.2.1.3 Download and Unzip the Patch : I assume this step has been done
4) Run OPatch Conflict Check:-
•For Grid Infrastructure Home, as home user: Change appropriate patch location
Login as grid
Login as grid
[grid@rac3 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/30920127/26839277
[grid@rac3 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/30920127/30882603
[grid@rac3 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/30920127/30886680
[grid@rac3 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/30920127/30888810
•For Database home, as home user:Change appropriate patch location
Login as oracle
Login as oracle
[oracle@rac3 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/30920127/30869447
[oracle@rac3 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/30920127/26839277
[oracle@rac3 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/30920127/30882603
[oracle@rac3 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/30920127/30886680
[oracle@rac3 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/30920127/30888810
Note:
When OPatch starts, it validates the patch and ensures that there are no conflicts with the software already installed in the ORACLE_HOME.
OPatch categorizes conflicts into the following types:
Conflicts with a patch already applied to the ORACLE_HOME.
In this case, stop the patch installation and contact Oracle Support Services.
Conflicts with subset patch already applied to the ORACLE_HOME.
In this case, continue with the patch installation because as the new patch contains all the fixes from the existing patch in the ORACLE_HOME.
And, in any case, the subset patch will automatically be rolled back prior to the installation of the new patch.
When OPatch starts, it validates the patch and ensures that there are no conflicts with the software already installed in the ORACLE_HOME.
OPatch categorizes conflicts into the following types:
Conflicts with a patch already applied to the ORACLE_HOME.
In this case, stop the patch installation and contact Oracle Support Services.
Conflicts with subset patch already applied to the ORACLE_HOME.
In this case, continue with the patch installation because as the new patch contains all the fixes from the existing patch in the ORACLE_HOME.
And, in any case, the subset patch will automatically be rolled back prior to the installation of the new patch.
5) Run OPatch SystemSpace Check:
Check if enough free space is available on the ORACLE_HOME filesystem for the patches to be applied as given below:
⇒For Grid Infrastructure Home, as home user:grid
===============================
1.Create file /tmp/patch_list_gihome.txt with the following content:
su - grid
cd /backup/PATCH/Apr2020
echo "/backup/PATCH/Apr2020/30920127/30869447" >> /backup/PATCH/Apr2020/patch_list_gihome.txt
echo "/backup/PATCH/Apr2020/30920127/26839277" >> /backup/PATCH/Apr2020/patch_list_gihome.txt
echo "/backup/PATCH/Apr2020/30920127/30888810" >> /backup/PATCH/Apr2020/patch_list_gihome.txt
echo "/backup/PATCH/Apr2020/30920127/30882603" >> /backup/PATCH/Apr2020/patch_list_gihome.txt
echo "/backup/PATCH/Apr2020/30920127/30886680" >> /backup/PATCH/Apr2020/patch_list_gihome.txt
cat /backup/PATCH/Apr2020/patch_list_gihome.txt
2.Run the opatch command to check if enough free space is available in the Grid Infrastructure Home:
$GRID_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Apr2020/patch_list_gihome.txt
⇨For Database home, as home user:
===============================
su - oracle
1.Create file /backup/PATCH/Apr2020/patch_list_dbhome.txt with the following content:
echo "/backup/PATCH/Apr2020/30920127/30882603" >> /backup/PATCH/Apr2020/patch_list_dbhome.txt
echo "/backup/PATCH/Apr2020/30920127/30886680" >> /backup/PATCH/Apr2020/patch_list_dbhome.txt
cat /backup/PATCH/Apr2020/patch_list_dbhome.txt
2. Run opatch command to check if enough free space is available in the Database Home:
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Apr2020/patch_list_dbhome.txt
The command output reports pass and fail messages as per the system space availability:
If OPatch reports Prereq "checkSystemSpace" failed., then cleanup the system space as the required amount of space is not available.
If OPatch reports Prereq "checkSystemSpace" passed., then no action is needed. Proceed with patch installation.
6. One-off Patch Conflict Detection and Resolution:-
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30869447 -analyze -oh $ORACLE_HOME
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/26839277 -analyze -oh $ORACLE_HOME
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30882603 -analyze -oh $ORACLE_HOME
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30886680 -analyze -oh $ORACLE_HOME
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30888810 -analyze -oh $ORACLE_HOME
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30888810 -analyze -oh $ORACLE_HOME
[root@rac3 ~]# export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/db_1
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30869447 -analyze -oh $ORACLE_HOME
oot@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/26839277 -analyze -oh $ORACLE_HOME
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30882603 -analyze -oh $ORACLE_HOME
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30886680 -analyze -oh $ORACLE_HOME
[root@rac3 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30888810 -analyze -oh $ORACLE_HOME
[root@rac4 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30869447 -analyze -oh $ORACLE_HOME
[root@rac4 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/26839277 -analyze -oh $ORACLE_HOME
[root@rac4 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30882603 -analyze -oh $ORACLE_HOME
[root@rac4 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30886680 -analyze -oh $ORACLE_HOME
[root@rac4 ~]# $ORACLE_HOME/OPatch/opatchauto apply /backup/30920127/30888810 -analyze -oh $ORACLE_HOME
Check all the CRS services on RAC3 and RAC4.
7. Patch Installation
RAC3 Instance :-
==>check status of database connection and distribution before starting patch or any other information like invalid objects etc , you can capture it at this point
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
run your scripts if any to gather any details before proceeding with the patch
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now shutdown the instance on the node-1, either you can use srvctl stop instance command or you can manually shutdown like below. I always prefer to manually shutdown instance like below:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
run your scripts if any to gather any details before proceeding with the patch
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now shutdown the instance on the node-1, either you can use srvctl stop instance command or you can manually shutdown like below. I always prefer to manually shutdown instance like below:
👉Patching GRID HOME and Oracle Database HOME Separately
Applying Patch on GRID_HOME - rac3 using root user
[root@rac3 ~]# /u02/app/12.2.0/grid/OPatch/opatchauto apply /backup/30920127 -oh /u02/app/12.2.0/grid
Apply patch on ORACLE_HOME as root user on RAC3
[root@rac3 ~]# /u01/app/oracle/product/12.2.0.1/db_1/OPatch/opatchauto apply /backup/30920127 -oh /u01/app/oracle/product/12.2.0.1/db_1
####>Start the Cluster service if not started automatically:
>>As ROOT user and make sure GRID_HOME is set
crsctl start crs
root@rac3:[/backup/PATCH]#echo $GRID_HOME
/u01/app/12.2.0.1/grid
root@rac3:[/backup/PATCH]#$GRID_HOME/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
root@rac3:[/backup/PATCH]#crsctl status res -t
>>To check status of Cluster services (all the CRS services are listed).
$crsctl status res -t
ps -ef|grep d.bin
sqlplus / as sysdba
>>>Take the lsinvntory details for DB and GRID homes.
opatch lsinventory -detail> grid_OPatch_lsinventory_detail_After_DBGrid_patch.lst
/u01/app/oracle/product/12.2.0/dbhome_1/OPatch/opatch lsinventory -detail >rdbms_OPatch_lsinventory_detail_After_DB_Patch.lst
==>start the instance:
[grid@rac3 ~]$ srvctl status database -d ORCL
Instance ORCL11 is not running on node rac3
Instance ORCL12 is running on node rac4
[grid@rac3 ~]$ srvctl start instance -d ORCL -i ORCL11
[grid@rac3 ~]$ srvctl status database -d ORCL
[grid@rac3 ~]$ srvctl status database -d ORCL
Instance ORCL11 is running on node rac3
Instance ORCL12 is running on node rac4
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
###>>Get the post Patch evidence from Node-1 After patch application:
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
Note: Perform this on both the nodes:
For Grid:
============
$ /u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_After.lst
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_detail_After.lst
###>>Get the post Patch evidence from Node-1 After patch application:
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
Note: Perform this on both the nodes:
For Grid:
============
$ /u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_After.lst
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_detail_After.lst
For RDBMS as oracle user:
======================
If this command succeeds, it lists the Oracle components that are installed in the home. Save the output so you have the status prior to the patch apply.
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_After.lst
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_detail_After.lst
~~~~~~~~completion of node-1 patching activity~~~~~~~~~~~~~~~~~~~~~~~~~~~~
======================
If this command succeeds, it lists the Oracle components that are installed in the home. Save the output so you have the status prior to the patch apply.
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_After.lst
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_detail_After.lst
~~~~~~~~completion of node-1 patching activity~~~~~~~~~~~~~~~~~~~~~~~~~~~~
###====>Perform the similar steps on Node-2:
==>1.2.4 Patch Installation ON NODE-2
########################################################################################
sqlplus / as sysdba
01:28:49 SQL> shut immediate;
Note: Post shutdown of the instance one, observed that connections/txns where moved slowly to node-2, after 10-15 min all connection(total 3593) were moved to node-2. Below is the output from instance -2:
Patching Oracle RAC Database Homes and GI Separately
•Case 1: Oracle RAC, where the GI Home and the Database Homes are not shared and ACFS file system is not configured.
oracle@:[/home/oracle]$ps -ef| grep sqlplus
oracle 9307060 7275390 0 03:13:07 pts/3 0:00 grep sqlplus
oracle@:[/home/oracle]$
########################################################################################
sqlplus / as sysdba
01:28:49 SQL> shut immediate;
Note: Post shutdown of the instance one, observed that connections/txns where moved slowly to node-2, after 10-15 min all connection(total 3593) were moved to node-2. Below is the output from instance -2:
Patching Oracle RAC Database Homes and GI Separately
•Case 1: Oracle RAC, where the GI Home and the Database Homes are not shared and ACFS file system is not configured.
oracle@:[/home/oracle]$ps -ef| grep sqlplus
oracle 9307060 7275390 0 03:13:07 pts/3 0:00 grep sqlplus
oracle@:[/home/oracle]$
==================Grid HOME:==================================
=============================================================
==> Please make sure to change the full permission otherwise patching may fail, due to file permission issue.
root@rac4:[/backup/PATCH]#chmod -R 775 /backup/30920127
-->Make sure that instance is down, This is optional but it's better to shutdown instance manually to make sure it is cleanly shutting down.
Login as oracle and shutdown instance. "Shutdown immediate;"
-->Now as a root User:
export GRID_HOME=/u01/app/12.2.0/grid <--Not Mandatory
echo $PATH
export PATH=/u01/app/12.2.0/grid/OPatch:$PATH
which opatch
opatch version
=============================================================
==> Please make sure to change the full permission otherwise patching may fail, due to file permission issue.
root@rac4:[/backup/PATCH]#chmod -R 775 /backup/30920127
-->Make sure that instance is down, This is optional but it's better to shutdown instance manually to make sure it is cleanly shutting down.
Login as oracle and shutdown instance. "Shutdown immediate;"
-->Now as a root User:
export GRID_HOME=/u01/app/12.2.0/grid <--Not Mandatory
echo $PATH
export PATH=/u01/app/12.2.0/grid/OPatch:$PATH
which opatch
opatch version
[root@rac4 ~]# /u02/app/12.2.0/grid/OPatch/opatchauto apply /backup/30920127 -oh /u02/app/12.2.0/grid
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u02/app/12.2.0/grid, host: rac4.
Command failed: /u02/app/12.2.0/grid/OPatch/opatchauto apply /backup/30920127 -oh /u02/app/12.2.0/grid -target_type cluster -binary -invPtrLoc
/u02/app/12.2.0/grid/oraInst.loc -jre /u02/app/12.2.0/grid/OPatch/jre -persistresult /u02/app/12.2.0/grid/opatchautocfg/db/sessioninfo/sessionresult_rac4_crs.ser
-analyzedresult /u02/app/12.2.0/grid/opatchautocfg/db/sessioninfo/sessionresult_analyze_rac4_crs.ser
Work Around
[root@rac4 ~]# /u02/app/12.2.0/grid/crs/install/rootcrs.sh -unlock
Using configuration parameter file: /u02/app/12.2.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/rac4/crsconfig/crsunlock_rac4_2021-07-16_00-39-33AM.log
2021/07/16 00:40:16 CLSRSC-4012: Shutting down Oracle Trace File Analyzer (TFA) Collector.
2021/07/16 00:40:30 CLSRSC-4013: Successfully shut down Oracle Trace File Analyzer (TFA) Collector.
2021/07/16 00:40:31 CLSRSC-347: Successfully unlock /u02/app/12.2.0/grid
[root@rac4 ~]# ps -ef | grep d.bin
root 57819 24816 0 00:40 pts/0 00:00:00 grep --color=auto d.bin
Now Applying the patch
[root@rac4 ~]# /u02/app/12.2.0/grid/OPatch/opatchauto apply /backup/30920127 -oh /u02/app/12.2.0/grid
==========================Oracle - DB===================
=========================================================
-------------------------------------------------------------------------
-->As a root user:
export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1 <-- Not mandatory
export PATH=/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch:$PATH
echo $PATH
which opatch
opatch version
=========================================================
-------------------------------------------------------------------------
-->As a root user:
export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1 <-- Not mandatory
export PATH=/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch:$PATH
echo $PATH
which opatch
opatch version
Apply patch on ORACLE_HOME as root user on RAC4
[root@rac3 ~]# /u01/app/oracle/product/12.2.0.1/db_1/OPatch/opatchauto apply /backup/30920127 -oh /u01/app/oracle/product/12.2.0.1/db_1
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u01/app/oracle/product/12.2.0.1/db_1, host: rac4.
Command failed: /u01/app/oracle/product/12.2.0.1/db_1/OPatch/opatchauto apply /backup/30920127 -oh /u01/app/oracle/product/12.2.0.1/db_1
-target_type rac_database -binary -invPtrLoc /u01/app/oracle/product/12.2.0.1/db_1/oraInst.loc -jre /u01/app/oracle/product/12.2.0.1/db_1/OPatch/jre -persistresult /u01/app/oracle/product/12.2.0.1/db_1/opatchautocfg/db/sessioninfo/sessionresult_rac4_rac.ser -analyzedresult /u01/app/oracle/product/12.2.0.1/db_1/opatchautocfg/db/sessioninfo/sessionresult_analyze_rac4_rac.ser
Command failure output:
==Following patches FAILED in apply:
Patch: /backup/30920127/30882603
Log: /u01/app/oracle/product/12.2.0.1/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-07-16_01-14-34AM_1.log
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: Re-link fails on target "install_srvm".
Workarounds
[oracle@rac4 lib]$ pwd
/u01/app/oracle/product/12.2.0.1/db_1/rdbms/lib
[oracle@rac4 lib]$ make -f ins_rdbms.mk ioracle
Now apply patching :-
[root@rac4 ~]# /u01/app/oracle/product/12.2.0.1/db_1/OPatch/opatchauto apply /backup/30920127 -oh /u01/app/oracle/product/12.2.0.1/db_1
~~~~~~~~~~~~~~~~~~~~~~~~~END of output of patch application on binary on node-2~~~~~~~~~~~~~~~~
####==>Datapatch application: only should be run from one node only.
==>Run as an oracle user
$] cd $ORACLE_HOME/OPatch
$]./datapatch -verbose
~~~~~~~~~ ==>o/p of :Datapatch application: only should be run from one node only.~~~~~~~~~~~~~~~~~~~~~~
==>Run as an oracle user
$] cd $ORACLE_HOME/OPatch
$]./datapatch -verbose
~~~~~~~~~ ==>o/p of :Datapatch application: only should be run from one node only.~~~~~~~~~~~~~~~~~~~~~~
oracle@rac3:[/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch]$./datapatch -verbose
SQL Patching tool version 12.2.0.1.0 Production on Sun Feb 2 15:21:24 2020
Copyright (c) 2012, 2020, Oracle. All rights reserved.
Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_34800046_2020_02_02_15_21_24/sqlpatch_invocation.log
Connecting to database...OK
Bootstrapping registry and package to current versions...done
Determining current state...done
Current state of SQL patches:
Bundle series DBRU:
ID 200414 in the binary registry and not installed in the SQL registry
Adding patches to installation queue and performing prereq checks...
Installation queue:
Nothing to roll back
The following patches will be applied:
30920127 (DATABASE JAN 2020 RELEASE UPDATE 12.2.0.1.200414)
Installing patches...
Patch installation complete. Total patches installed: 1
Validating logfiles...
Patch 30920127 apply:
logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30920127/23339232/30920127_apply_ORCL_2020Feb02_15_22_57.log
Adding patches to retry installation queue...
Retry installation queue:
Nothing to roll back
The following patches will be applied:
30920127 (DATABASE JAN 2020 RELEASE UPDATE 12.2.0.1.200414)
Installing patches...
Patch installation complete. Total patches installed: 1
Validating logfiles...
Patch 30920127 apply: SUCCESS
logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30920127/23339232/30920127_apply_ORCL_2020Feb02_16_11_06.log (no errors)
SQL Patching tool complete on Sun Feb 2 16:21:59 2020
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~End of datapatch application~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~>Check the status of datapatch/modified SQL
set line 500
col patch_id for 9999999999
col patch_uid for 9999999999
col version for a10
col action for a10
col install_id for 99
col action_time for a30
col description for a60
col bundle_series for a20
col status for a30
select patch_id, patch_uid, version, action, install_id, action_time, description, bundle_series,status from dba_registry_sqlpatch;
~~>Check the status of datapatch/modified SQL
set line 500
col patch_id for 9999999999
col patch_uid for 9999999999
col version for a10
col action for a10
col install_id for 99
col action_time for a30
col description for a60
col bundle_series for a20
col status for a30
select patch_id, patch_uid, version, action, install_id, action_time, description, bundle_series,status from dba_registry_sqlpatch;
SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> select patch_id, patch_uid, version, action, install_id, action_time, description, bundle_series,status from dba_registry_sqlpatch;
PATCH_ID PATCH_UID VERSION ACTION INSTALL_ID ACTION_TIME DESCRIPTION BUNDLE_SERIES STATUS
----------- ----------- ---------- ---------- ---------- ------------------------------ ------------------------------------------------------------ -------------------- ------------------------------
30886680 23474251 12.2.0.1 APPLY 1 16-JUL-21 11.43.36.541208 AM DATABASE APR 2020 RELEASE UPDATE 12.2.0.1.200414 DBRU SUCCESS
SQL>
No comments:
Post a Comment