GRID_HOME: /u01/app/11.2.0/grid
ORACLE_HOME: /u01/app/oracle/product/11.2.0/dbhome_1
- Patching RAC as a single instance (All-Node Patch)
- Patching RAC using a minimum down-time strategy (Min. Downtime Patch)
- Patching RAC using a rolling strategy - No down time (Rolling Patch)
- Patching RAC as a single instance (All-Node Patch)
- Shutdown all Oracle instances on all nodes
- Apply the patch to the RAC home on all nodes
- Bring all instances up
- Patching RAC using a minimum down-time strategy (Min. Downtime Patch)
- Shutdown all the Oracle instances on node 1
- Apply the patch to the RAC home on node 1
- Shutdown all the Oracle instances on node 2
- Apply the patch to the RAC home on node 2
- Shutdown all the Oracle instances on node 3
- At this point, instances on nodes 1 and 2 can be brought up
- Apply the patch to the RAC home on node 3
- Startup all the Oracle instances on node 3
- Patching RAC using a rolling strategy - No down time (Rolling Patch)
- Shutdown all the Oracle instances on node 1
- Apply the patch to the RAC home on node 1
- Start all the Oracle instances on node 1
- Shutdown all the Oracle instances on node 2
- Apply the patch to the RAC home on node 2
- Start all the Oracle instances on node 2
- Shutdown all the Oracle instances on node 3
- Apply the patch to the RAC home on node 3
- Start all the Oracle instances on node 3
If (users specify minimize_downtime)
patching mechanism = Min. Downtime
else if (patch is a rolling patch)
patching mechanism = Rolling
else
patching mechanism = All-Node
- Patches that do not affect the contents of the database.
- Patches that are not related to the RAC internode communication infrastructure.
- Patches that change procedural logic and do not modify common header definitions of kernel modules.
- This includes client side patches that only affect utilities like export, import, sql*plus, sql*loader, etc.
How to determine if a patch is a "rolling patch" or not?
- 9i or 10gR1: opatch query -is_rolling
- 10gR2: opatch query -all [unzipped patch location] | grep rolling
- 10gR2 on Windows: opatch query -all [unzipped patch location] | findstr rolling
- Later 10gR2 or 11g: opatch query -is_rolling_patch [unzipped patch location]
Released: Apr 18, 2017
Some Important Notes:
- Patch can be applied to both Oracle Database Home refers to Enterprise Edition or Standard Edition Database software.
- The GI PSU patch includes updates for both the Clusterware home and Database home that can be applied in a rolling fashion.
- This patch is Database Vault installable.
- This patch is Data Guard Standby First Installable.
- Data Guard Standby-First Patch Apply for details on how to remove risk and reduce downtime when applying this patch.
- GI PSUs are cumulative and include the Database PSU and associated CPU program security content.
- To get total number of servers to be patched.
- To download a patch.
- To upload the patch to target servers.
- To perform prerequisites.
- To confirm client if you find any conflict in prerequisites.
- To inform client that we are ready for patching if no conflict found.
unzip p25476126_112040_<platform>.zip
Step 4: Stop Oracle services
The below command will stop all Oracle services corresponds to mentioned Oracle Home.
Please do not delete status.log file which will be required later to start the service.
By oracle user:
$srvctl stop home -o $ORACLE_HOME -s status.log -n rac1
$ps –ef | grep pmon
$ps –ef | grep tns
$srvctl status database –d DC
$ps –ef | grep ORA_
#/u01/app/11.2.0/grid/crs/install/rootcrs.pl -unlock
The above script will perform the following actions before patching:
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
· Shutdown of Oracle High Availability Services-managed resources on 'rac1'
· To stop 'ora.crsd' on 'rac1'
· Shutdown of Cluster Ready Services-managed resources on 'rac1'
· To stop 'ora.LISTENER_SCAN3.lsnr' on 'rac1'
· To stop 'ora.LISTENER.lsnr' on 'rac1'
· To stop 'ora.registry.acfs' on 'rac1'
· To stop 'ora.DATA.dg' on 'rac1'
· To stop 'ora.oc4j' on 'rac1'
· To stop 'ora.LISTENER_SCAN2.lsnr' on 'rac1'
· To stop 'ora.cvu' on 'rac1'
· Stop of 'ora.cvu' on 'rac1' succeeded
· To start 'ora.cvu' on 'rac2'
· Stop of 'ora.LISTENER_SCAN3.lsnr' on 'rac1' succeeded
· To stop 'ora.scan3.vip' on 'rac1'
· Stop of 'ora.LISTENER.lsnr' on 'rac1' succeeded
· To stop 'ora.rac1.vip' on 'rac1'
· Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rac1' succeeded
· To stop 'ora.scan2.vip' on 'rac1'
· Start of 'ora.cvu' on 'rac2' succeeded
· Stop of 'ora.scan3.vip' on 'rac1' succeeded
· To start 'ora.scan3.vip' on 'rac2'
· Stop of 'ora.registry.acfs' on 'rac1' succeeded
· Stop of 'ora.rac1.vip' on 'rac1' succeeded
· Start 'ora.rac1.vip' on 'rac2'
· Stop of 'ora.scan2.vip' on 'rac1' succeeded
· To start 'ora.scan2.vip' on 'rac2'
· Start of 'ora.scan3.vip' on 'rac2' succeeded
· To start 'ora.LISTENER_SCAN3.lsnr' on 'rac2'
· Start of 'ora.rac1.vip' on 'rac2' succeeded
· Start of 'ora.scan2.vip' on 'rac2' succeeded
· To start 'ora.LISTENER_SCAN2.lsnr' on 'rac2'
· Stop of 'ora.oc4j' on 'rac1' succeeded
· To start 'ora.oc4j' on 'rac2'
· Start of 'ora.LISTENER_SCAN3.lsnr' on 'rac2' succeeded
· Stop of 'ora.DATA.dg' on 'rac1' succeeded
· To stop 'ora.asm' on 'rac1'
· Stop of 'ora.asm' on 'rac1' succeeded
· Start of 'ora.oc4j' on 'rac2' succeeded
· Start of 'ora.LISTENER_SCAN2.lsnr' on 'rac2' succeeded
· To stop 'ora.ons' on 'rac1'
· Stop of 'ora.ons' on 'rac1' succeeded
· To stop 'ora.net1.network' on 'rac1'
· Stop of 'ora.net1.network' on 'rac1' succeeded
· Shutdown of Cluster Ready Services-managed resources on 'rac1' has completed
· Stop of 'ora.crsd' on 'rac1' succeeded
· To stop 'ora.drivers.acfs' on 'rac1'
· To stop 'ora.crf' on 'rac1'
· To stop 'ora.ctssd' on 'rac1'
· To stop 'ora.evmd' on 'rac1'
· To stop 'ora.asm' on 'rac1'
· To stop 'ora.mdnsd' on 'rac1'
· Stop of 'ora.crf' on 'rac1' succeeded
· Stop of 'ora.evmd' on 'rac1' succeeded
· Stop of 'ora.mdnsd' on 'rac1' succeeded
· Stop of 'ora.ctssd' on 'rac1' succeeded
· Stop of 'ora.asm' on 'rac1' succeeded
· To stop 'ora.cluster_interconnect.haip' on 'rac1'
· Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
· To stop 'ora.cssd' on 'rac1'
· Stop of 'ora.cssd' on 'rac1' succeeded
· To stop 'ora.gipcd' on 'rac1'
· Stop of 'ora.drivers.acfs' on 'rac1' succeeded
· Stop of 'ora.gipcd' on 'rac1' succeeded
· To stop 'ora.gpnpd' on 'rac1'
· Stop of 'ora.gpnpd' on 'rac1' succeeded
· Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
· CRS-4133: Oracle High Availability Services has been stopped.
· Successfully unlock /u01/app/11.2.0/grid
Step 8: Run pre-script for Database Home
Run the below command by Oracle User:
/home/oracle/25476126/23054319/custom/server/23054319/custom/scripts/prepatch.sh completed successfully.
$/u01/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch napply -oh /u01/app/oracle/product/11.2.0/dbhome_1 –local /home/oracle/25476126/23054319/custom/server/23054319
$/u01/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch apply -oh /u01/app/oracle/product/11.2.0/dbhome_1 -local /home/oracle/25476126/24732075
$/home/oracle/25476126/23054319/custom/server/23054319/custom/scripts/postpatch.sh -dbhome /u01/app/oracle/product/11.2.0/dbhome_1
Reading /u01/app/oracle/product/11.2.0/dbhome_1/install/params.ora..
Reading /u01/app/oracle/product/11.2.0/dbhome_1/install/params.ora..
Parsing file /u01/app/oracle/product/11.2.0/dbhome_1/bin/racgwrap
Parsing file /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvctl
Parsing file /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvconfig
Parsing file /u01/app/oracle/product/11.2.0/dbhome_1/bin/cluvfy
Verifying file /u01/app/oracle/product/11.2.0/dbhome_1/bin/racgwrap
Verifying file /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvctl
Verifying file /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvconfig
Verifying file /u01/app/oracle/product/11.2.0/dbhome_1/bin/cluvfy
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/racgwrap
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvctl
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/srvconfig
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/cluvfy
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/diskmon.bin
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/lsnodes
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/osdbagrp
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/rawutl
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/srvm/admin/ractrans
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/srvm/admin/getcrshome
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/gnsd
Reapplying file permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin/crsdiag.pl
Postpatch completed successfully
By Root User:
#/u01/app/11.2.0/grid/rdbms/install/rootadd_rdbms.sh
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Installing Trace File Analyzer
CRS-4123: Oracle High Availability Services has been started.
The below command will start all Oracle services using status.log file corresponds to mentioned Oracle Home.
By oracle user:
$srvctl start home -o $ORACLE_HOME -s status.log -n rac1
$ps –ef | grep pmon
$ps –ef | grep tns
$srvctl status database –d DC
$ps –ef | grep ORA_
$srvctl stop home -o $ORACLE_HOME -s status.log -n rac2
$srvctl start home -o $ORACLE_HOME -s status.log -n rac2
catbundle.sql
execution is reflected in the dba_registry_history
view by a row associated
with bundle series PSU
.cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle.sql psu apply
SQL> QUIT
Verify below logfiles for any errors.
catbundle_PSU_<database SID>_GENERATE_<TIMESTAMP>.log
$ rman catalog username/password@alias
RMAN> UPGRADE CATALOG;
RMAN> UPGRADE CATALOG;
You have successfully applied latest PSU patch to both Grid and Oracle Homes.
++++++++++++++++++++++++ END +++++++++++++++++++++++++++++++++++++
Rollback Plan
Step 1: Stop Oracle services
The below command will stop all Oracle services corresponds to mentioned Oracle Home.
Please do not delete status.log file which will be required later to start the service.
By oracle user:
$srvctl stop home -o $ORACLE_HOME -s status.log -n rac1
$ps –ef | grep pmon
$ps –ef | grep tns
$srvctl status database –d DC
$ps –ef | grep ORA_
#/u01/app/11.2.0/grid/crs/install/rootcrs.pl -unlock
$ /u01/app/11.2.0/grid/OPatch/opatch rollback -local -id 23054319 -oh /u01/app/11.2.0/grid
$ /u01/app/11.2.0/grid OPatch/opatch rollback -local -id 24732075 -oh /u01/app/11.2.0/grid
$opatch rollback -local -id 23054319 -oh /u01/app/oracle/product/11.2.0/dbhome_1
$opatch rollback -local -id 24732075 -oh /u01/app/oracle/product/11.2.0/dbhome_1
$/home/grid/25476126/23054319/custom/server/23054319/custom/scripts/postpatch.sh -dbhome /u01/app/oracle/product/11.2.0/dbhome_1
#/u01/app/11.2.0/grid/rdbms/install/rootadd_rdbms.sh
#/u01/app/11.2.0/grid/crs/install/rootcrs.pl -patch
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Installing Trace File Analyzer
CRS-4123: Oracle High Availability Services has been started.
The below command will start all Oracle services using status.log file corresponds to mentioned Oracle Home.
By oracle user:
$srvctl start home -o $ORACLE_HOME -s status.log -n rac1
$ps –ef | grep pmon
$ps –ef | grep tns
$srvctl status database –d DC
$ps –ef | grep ORA_
$srvctl stop home -o $ORACLE_HOME -s status.log -n rac1
ORACLE_HOME
,
connect to the database using SQL*Plus as SYSDBA
and run the rollback script as follows:cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle_PSU_<database SID PREFIX>_ROLLBACK.sql
SQL> QUIT
$ opatch lsinventory
Hello Guys,
Please comment if you like this post.
Wow !! very nice Rupesh, Thanks for posting this Nicely making steps.
ReplyDeletevery nice and clear steps..!
ReplyDeletethanks,
Siva
Sir, Could you please explain how to apply patch on two node RAC 12C
ReplyDeleteThanks for your comment.
ReplyDeleteI will upload document on this soon.
Good one..Thanks for the details steps.
ReplyDeletevery use full block all steps are very clear..thanks for sharing this knoweleadge
ReplyDeleteIndeed its crystal clear..great job!!!
ReplyDeleteRupesh sir really nice to explain each and every steps and all concept regarding patch was very cleared. Thanks once again Rupesh Sir.
ReplyDeleteHi Rupesh,
ReplyDeleteCould you pls explain what is mean by
"this document is for Non-Shared Cluster Environment. For Cluster shared environment, there is a different method to apply patch. "
what is the difference between Non-Shared Cluster vs Cluster shared environment ???
I always welcome for queries.
DeleteIf you go to readme file, there are different methods of applying patches for rac dbs for shared file systems and non- shared file systems. Shared file system means Oracle ACFS cluster file system. If you are using ACFS then you have to use different method to apply patch.