Oracle DBA's : Grid Upgrade 12.2.0.1 to 18.3.0.0

Upgrading Oracle Grid Infrastructure

Oracle Grid Upgrade 12.2.0.1 to 18c

Upgrading to Oracle Database 18c

Upgrade Grid Infrastructure to 18c

STEP 1 Grid Upgrade:

Oracle Grid upgrade from 12g to 18c

PURPOSE: All documents are provided on this Blog just for educational purposes only. Please make sure that you run it in your test environment before to move on to production environment.

Preparation:

Keep following detail for later

srvctl config asm

ASM home: <CRS home>

Password file:

Backup of Password file:

ASM listener: LISTENER

Spfile: +DATA/asm/asmparameterfile/registry.253.1024325155

ASM diskgroup discovery string: /dev/*

We need to download following Files:

LINUX.X64_180000_grid_home.zip

p28096386_180000_Linux-x86-64.zip

Create NEW directory for NEW Oracle Grid home:

$ORACLE_BASE/product/18.0.0/grid_18c

mv /download/LINUX.X64_180000_grid_home.zip $ORACLE_BASE/product/18.0.0/grid_18c

unzip LINUX.X64_180000_grid_home.zip

Stop ASM and DB

SHUTDOWN DB FROM ASM HOME

srvctl status database -d KARDBPRD

Database is running.

srvctl stop database -d KARDBPRD

srvctl status listener -l LISTENER

srvctl stop listener -l LISTENER

srvctl status asm

ASM is running on kardbprd.us.com

srvctl stop asm -f

PRCR-1065 : Failed to stop resource ora.asm

CRS-2675: Stop of 'ora.DATA.dg' on 'kardbprd.us.com' failed

The workaround is to shutdown ASM using sqlplus(Doc ID 2271669.1)

sqlplus -s / as sysasm

shutdown immediate

Stop complete Restart stack with below command.

$crsctl stop has

crsctl stat res -t

Precheck:

/apps/oracle/product/18.0.0/grid_18c/runcluvfy.sh stage -pre hacfg

This is the output of the above command:

Verifying Physical Memory ...PASSED

Verifying Available Physical Memory ...PASSED

Verifying Swap Size ...PASSED

Verifying Free Space: kardbprd.us.com:/usr,kardbprd.us.com:/var,kardbprd.us.com:/etc,kardbprd.us.com:/sbin,kardbprd.us.com:/tmp ...PASSED

Verifying User Existence: oracle ...

Verifying Users With Same UID: 1001 ...PASSED

Verifying User Existence: oracle ...PASSED

Verifying Group Existence: dba ...PASSED

Verifying Group Membership: dba ...PASSED

Verifying Run Level ...PASSED

Verifying Architecture ...PASSED

Verifying OS Kernel Version ...PASSED

Verifying OS Kernel Parameter: semmsl ...PASSED

Verifying OS Kernel Parameter: semmns ...PASSED

Verifying OS Kernel Parameter: semopm ...PASSED

Verifying OS Kernel Parameter: semmni ...PASSED

Verifying OS Kernel Parameter: shmmax ...PASSED

Verifying OS Kernel Parameter: shmmni ...PASSED

Verifying OS Kernel Parameter: shmall ...PASSED

Verifying OS Kernel Parameter: file-max ...PASSED

Verifying OS Kernel Parameter: ip_local_port_range ...PASSED

Verifying OS Kernel Parameter: rmem_default ...PASSED

Verifying OS Kernel Parameter: rmem_max ...PASSED

Verifying OS Kernel Parameter: wmem_default ...PASSED

Verifying OS Kernel Parameter: wmem_max ...PASSED

Verifying OS Kernel Parameter: aio-max-nr ...PASSED

Verifying OS Kernel Parameter: panic_on_oops ...PASSED

Verifying Package: binutils-2.20.51.0.2 ...PASSED

Verifying Package: compat-libcap1-1.10 ...PASSED

Verifying Package: compat-libstdc++-33-3.2.3 (x86_64) ...PASSED

Verifying Package: libgcc-4.4.7 (x86_64) ...PASSED

Verifying Package: libstdc++-4.4.7 (x86_64) ...PASSED

Verifying Package: libstdc++-devel-4.4.7 (x86_64) ...PASSED

Verifying Package: sysstat-9.0.4 ...PASSED

Verifying Package: gcc-4.4.7 ...PASSED

Verifying Package: gcc-c++-4.4.7 ...PASSED

Verifying Package: ksh ...PASSED

Verifying Package: make-3.81 ...PASSED

Verifying Package: glibc-2.12 (x86_64) ...PASSED

Verifying Package: glibc-devel-2.12 (x86_64) ...PASSED

Verifying Package: libaio-0.3.107 (x86_64) ...PASSED

Verifying Package: libaio-devel-0.3.107 (x86_64) ...PASSED

Verifying Package: nfs-utils-1.2.3-15 ...PASSED

Verifying Package: smartmontools-5.43-1 ...PASSED

Verifying Package: net-tools-1.60-110 ...PASSED

Verifying Users With Same UID: 0 ...PASSED

Verifying Current Group ID ...PASSED

Verifying Root user consistency ...PASSED

Pre-check for Oracle Restart configuration was successful.

CVU operation performed: stage -pre hacfg

Date: Nov 25, 2019 10:09:28 AM

CVU home: /apps/oracle/product/18.0.0/grid_18c/

User: oracle

Unzip the patch which is required for Installation.

unzip p28096386_180000_Linux-x86-64.zip

As root user:

# cd $ORACLE_BASE/product/18.0.0/grid_18c/cv/rpm

# rpm -Uvh cvuqdisk-1.0.10-1.rpm

Preparing... ########################################### [100%]

1:cvuqdisk ########################################### [100%]

STARTING with Upgrade

I am using MobaXterm for X display

unset ORACLE_BASE

unset ORACLE_HOME

unset ORACLE_SID

DISPLAY=xxx.xx.xxx.xxx:0.0; export DISPLAY

./gridSetup.sh -applyRU /download/28096386

NOTE: I am not going upload the screenshot. Since the steps are pretty straightforward. I chose following options.

=>> Upgrade Orcle Grid Infrastructure

=>> Registration with Enterprise Manager (EM) Cloud Control =>> no need uncheck it

=>> Oracle_base (keep the same location)

=>> Automatically run configuration scripts =>> no need uncheck it

As root user:

/apps/oracle/product/18.0.0/grid_18c/rootupgrade.sh

I am getting following error:

/apps/oracle/crsdata/kardbprd.us.com/crsconfig/roothas_2019-11-25_10-38-40AM.log

2019/11/25 10:38:40 CLSRSC-595: Executing upgrade step 1 of 12: 'UpgPrechecks'.

2019/11/25 10:38:42 CLSRSC-595: Executing upgrade step 2 of 12: 'GetOldConfig'.

2019/11/25 10:38:58 CLSRSC-318: Failed to start Oracle OHASD service

Died at /apps/oracle/product/18.0.0/grid_18c/crs/install/crsupgrade.pm line 6104.

After debugging I realized I need to start the process manually

crsctl start has

CRS-4123: Oracle High Availability Services has been started.

Executing rootupgrade.sh script again and got another error :

/apps/oracle/product/18.0.0/grid_18c/rootupgrade.sh

CLSRSC-164: ASM upgrade failed

CLSRSC-304: Failed to upgrade ASM for Oracle Restart configuration

I have to start ASM via following command in a different screen.

srvctl start asm

Executing rootupgrade.sh script again:

/apps/oracle/product/18.0.0/grid_18c/rootupgrade.sh

It Works!

I just want to make sure that everything works properly after upgrade. I am going stop everything and will start again.

srvctl status asm

crsctl stat res -t

srvctl stop asm -f

crsctl stop has

crsctl start has

srvctl start asm

Adjust the location and any parameter required in the tnsnames.ora file

srvctl stop listener -l LISTENER

srvctl start listener -l LISTENER

Stop/Start DB from the DB_HOME

srvctl stop database -d KARDBPRD

srvctl start database -d KARDBPRD

Login into DB and in ASM Instance and verify the standard configurations.

$ sqlplus / as sysasm

SQL*Plus: Release 18.0.0.0.0 - Production on Mon Nov 25 13:38:48 2019

Version 18.3.0.0.0

Connected to:

Oracle Database 18c Enterprise Edition Release 18.0.0.0.0 - Production

Version 18.3.0.0.0

Change the compatibility parameter:

sqlplus / as sysasm

col COMPATIBILITY form a10

col DATABASE_COMPATIBILITY form a10

col NAME form a20

select group_number, name, compatibility, database_compatibility from v$asm_diskgroup;

GROUP_NUMBER NAME COMPATIBIL DATABASE_C

------------ -------------------- ---------- ----------

1 DATA 11.2.0.0.0 10.1.0.0.0

asmcmd setattr -G DATA compatible.asm 12.2.0.0.0

ALTER DISKGROUP DATA SET ATTRIBUTE 'compatible.asm' = '12.2.0.0.0'

SQL> select group_number, name, compatibility, database_compatibility from v$asm_diskgroup;

GROUP_NUMBER NAME COMPATIBIL DATABASE_C

------------ -------------------- ---------- ----------

1 DATA 12.2.0.0.0 10.1.0.0.0

Login to the DB:

sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Mon Nov 25 13:30:02 2019

Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> select name from v$database;

NAME

---------

KARDBPRD

Looks everything good. At this point my Grid ASM is running to the version 18.3.0 and my DB has 12.2.0.1 Version.

Let us upgrade DB now:

STEP 2 Database Upgrade:

Oracle database upgrade from 12g to 18c

1. Login as 12c DB

SQL> create pfile='/download/initDB_12c.ora' from spfile;

NOTE: Do not change this parameter keep as it is. You can do later.

*.compatible

I am using the GUI for Installation. Install Just Software.

We need following files. Download it and move or copy the file to the NEW $ORACLE_HOME

mv LINUX.X64_180000_db_home.zip /apps/oracle/product/18.0.0/db_18c

cd $ORACLE_HOME

unzip LINUX.X64_180000_db_home.zip

2. Install just Software.

export DISPLAY=xx.xx.xx.xx:0.0

unset ORACLE_BASE

unset ORACLE_HOME

unset ORACLE_SID

./runInstaller

Chose following options:

=>> Set Up Software Only

=>> Single Instance database installation

=>> Enterprise Edition

=>> chose the group you have. I chose “dba”

=>> Install

Execute the root script as a root user.

/apps/oracle/product/18.0.0/db_18c/root.sh

This is the output of the root.sh script:

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /apps/oracle/product/18.0.0/db_18c

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Do you want to setup Oracle Trace File Analyzer (TFA) now ? yes|[no] :

Oracle Trace File Analyzer (TFA - Non Daemon Mode) is available at :

/apps/oracle/product/18.0.0/db_18c/suptools/tfa/release/tfa_home/bin/tfactl

Note :

1. tfactl will use TFA Daemon Mode if TFA already running in Daemon Mode and user has access to TFA

2. tfactl will configure TFA Non Daemon Mode only if user has no access to TFA Daemon mode or TFA Daemon mode is not installed

Oracle Trace File Analyzer (TFA - Daemon Mode) can be installed by running this script :

/apps/oracle/product/18.0.0/db_18c/suptools/tfa/release/tfa_home/install/roottfa.sh

3. Execute preupgrade script as follow.

apps/oracle/product/12.2.0.1/db_12c/jdk/bin/java -jar /apps/oracle/product/18.0.0/db_18c/rdbms/admin/preupgrade.jar FILE TEXT

==================

PREUPGRADE SUMMARY

==================

/apps/oracle/cfgtoollogs/KARDBPRD/preupgrade/preupgrade.log

/apps/oracle/cfgtoollogs/KARDBPRD/preupgrade/preupgrade_fixups.sql

/apps/oracle/cfgtoollogs/KARDBPRD/preupgrade/postupgrade_fixups.sql

Execute fixup scripts as indicated below:

Before upgrade log into the database and execute the preupgrade fixups

@/apps/oracle/cfgtoollogs/KARDBPRD/preupgrade/preupgrade_fixups.sql

After the upgrade:

Log into the database and execute the postupgrade fixups

@/apps/oracle/cfgtoollogs/KARDBPRD/preupgrade/postupgrade_fixups.sql

Preupgrade complete: 2019-11-26T15:14:48

4. Run 12.2.0.1.0 $ORACLE_HOME/rdbms/admin/utlrp.sql to recompile invalid

@/apps/oracle/product/12.2.0.1/db_12c/rdbms/admin/utlrp.sql

5. Run the preupgrade_fixups.sql script

@/oracle/cfgtoollogs/KARDBPRD/preupgrade/preupgrade_fixups.sql

Check invalid Objects:

select comp_id,status from dba_registry;

Stop DB:

srvctl stop database -d KARDBPRD

srvctl stop listener -l LISTENER

Login as 18c:

create spfile from pfile='/download/initDB_12c.ora';

SQL> startup upgrade;

ORACLE instance started.

Total System Global Area 3.0266E+10 bytes

Fixed Size 12214256 bytes

Variable Size 3623878656 bytes

Database Buffers 2.6575E+10 bytes

Redo Buffers 54894592 bytes

Database mounted.

Database opened.

Execute following command to start the Update:

Start the database upgrade in parallel as follow. Or you can use "dbupgrade" command which is a wrapper for $ORACLE_HOME/perl/bin/perl catctl.pl -n 8 catupgrd.sql:

I am using the command as follow:

cd $ORACLE_HOME/rdbms/admin

$ORACLE_HOME/perl/bin/perl catctl.pl -n 8 catupgrd.sql

...

------------------------------------------------------

Phases [0-108] End Time:[2019_11_26 17:00:36]

------------------------------------------------------

Grand Total Time: 783s

LOG FILES: (/apps/oracle/product/18.0.0/db_18c/cfgtoollogs/KARDBPRD/upgrade20191126164734/catupgrd*.log)

Upgrade Summary Report Located in:

/apps/oracle/product/18.0.0/db_18c/cfgtoollogs/KARDBPRD/upgrade20191126164734/upg_summary.log

Grand Total Upgrade Time: [0d:0h:13m:3s]

Add DB to the NEW Oracle_Home via srvctl command:

srvctl add database -d KARDBPRD -o /apps/oracle/product/18.0.0/db_18c

srvctl start database -d KARDBPRD

Execute the postupgrade script.

@/apps/oracle/cfgtoollogs/KARDBPRD/preupgrade/postupgrade_fixups.sql

EXECUTE DBMS_STATS.GATHER_DICTIONARY_STATS;

EXECUTE DBMS_STATS.GATHER_FIXED_OBJECTS_STATS;

Check Components:

col COMP_NAME format a50;

set linesize 300

set pagesize 100

select COMP_NAME, version, status from dba_registry;SQL> SQL> SQL>

COMP_NAME VERSION STATUS

-------------------------------------------------- ------------------------------ -----------

Oracle Database Catalog Views 18.0.0.0.0 UPGRADED

Oracle Database Packages and Types 18.0.0.0.0 UPGRADED

JServer JAVA Virtual Machine 18.0.0.0.0 VALID

Oracle XDK 18.0.0.0.0 UPGRADED

Oracle Database Java Packages 18.0.0.0.0 UPGRADED

Oracle Real Application Clusters 18.0.0.0.0 OPTION OFF

Oracle Workspace Manager 18.0.0.0.0 UPGRADED

Oracle XML Database 18.0.0.0.0 UPGRADED

Oracle Multimedia 18.0.0.0.0 UPGRADED

Oracle Application Express 18.2.0.00.12 INVALID

10 rows selected.

SQL> select VERSION,name,open_mode from v$instance,v$database;

VERSION NAME OPEN_MODE

----------------- --------- --------------------

18.0.0.0.0 KURSDB READ WRITE

Checking TimeZone Version, which is also need to update:

SQL> SELECT * FROM v$timezone_file;

FILENAME VERSION CON_ID

-------------------- ---------- ----------

timezlrg_14.dat 14 0

SQL> select TZ_VERSION from registry$database;

TZ_VERSION

----------

====================

Update the timezone

====================

You may need to update the timezone as follow:

You need following commands:

@?/rdbms/admin/utltz_countstar.sql

@?/rdbms/admin/utltz_upg_check.sql

@?/rdbms/admin/utltz_upg_apply.sql

SQL> @utltz_upg_check.sql

Session altered.

INFO: Starting with RDBMS DST update preparation.

INFO: NO actual RDBMS DST update will be done by this script.

INFO: If an ERROR occurs the script will EXIT sqlplus.

INFO: Doing checks for known issues ...

INFO: Database version is 18.0.0.0 .

INFO: Database RDBMS DST version is DSTv14 .

INFO: No known issues detected.

INFO: Now detecting new RDBMS DST version.

A prepare window has been successfully started.

INFO: Newest RDBMS DST version detected is DSTv31 .

INFO: Next step is checking all TSTZ data.

INFO: It might take a while before any further output is seen ...

A prepare window has been successfully ended.

INFO: A newer RDBMS DST version than the one currently used is found.

INFO: Note that NO DST update was yet done.

INFO: Now run utltz_upg_apply.sql to do the actual RDBMS DST update.

INFO: Note that the utltz_upg_apply.sql script will

INFO: restart the database 2 times WITHOUT any confirmation or prompt.

Session altered.

SQL> @utltz_upg_apply.sql

SQL> SELECT * FROM v$timezone_file;

FILENAME VERSION CON_ID

-------------------- ---------- ----------

timezlrg_31.dat 31 0

1 row selected.

SQL> select TZ_VERSION from registry$database;

TZ_VERSION

----------

At this point we are done:

The best person is the one who benefits all human beings. [Prophet Muhammad (PBUH)]

This is the COMPLETE OUTPUT FROM THE rootupgrade.sh Script:

/apps/oracle/product/18.0.0/grid_18c/rootupgrade.sh

Performing root user operation.

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /apps/oracle/product/18.0.0/grid_18c

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /apps/oracle/product/18.0.0/grid_18c/crs/install/crsconfig_params

The log of current session can be found at:

/apps/oracle/crsdata/kardbprd.us.com/crsconfig/roothas_2019-11-25_12-45-31AM.log

2019/11/25 12:45:32 CLSRSC-595: Executing upgrade step 1 of 12: 'UpgPrechecks'.

2019/11/25 12:45:34 CLSRSC-595: Executing upgrade step 2 of 12: 'GetOldConfig'.

2019/11/25 12:45:35 CLSRSC-595: Executing upgrade step 3 of 12: 'GenSiteGUIDs'.

2019/11/25 12:45:35 CLSRSC-595: Executing upgrade step 4 of 12: 'SetupOSD'.

2019/11/25 12:45:35 CLSRSC-595: Executing upgrade step 5 of 12: 'PreUpgrade'.

ASM has been upgraded and started successfully.

2019/11/25 12:46:13 CLSRSC-595: Executing upgrade step 6 of 12: 'UpgradeAFD'.

2019/11/25 12:46:13 CLSRSC-595: Executing upgrade step 7 of 12: 'UpgradeOLR'.

Creating OCR keys for user 'oracle', privgrp 'dba'..

Operation successful.

2019/11/25 12:46:19 CLSRSC-595: Executing upgrade step 8 of 12: 'UpgradeOCR'.

LOCAL ONLY MODE

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

CRS-4664: Node kardbprd.us.com successfully pinned.

2019/11/25 12:46:20 CLSRSC-595: Executing upgrade step 9 of 12: 'CreateOHASD'.

2019/11/25 12:46:21 CLSRSC-595: Executing upgrade step 10 of 12: 'ConfigOHASD'.

2019/11/25 12:46:21 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.conf'

2019/11/25 12:46:34 CLSRSC-595: Executing upgrade step 11 of 12: 'UpgradeSIHA'.

CRS-4123: Oracle High Availability Services has been started.

kardbprd.us.com 2019/11/25 12:47:05 /apps/oracle/product/18.0.0/grid_18c/cdata/kardbprd.us.com/backup_20191125_124705.olr 70732493

kardbprd.us.com 2019/11/19 14:47:54 /apps/oracle/product/12.2.0.1/grid_12c/cdata/kardbprd.us.com/backup_20191119_144754.olr 1039292241

kardbprd.us.com 2019/11/14 14:44:41 /apps/oracle/product/11.2.0.3/grid/cdata/kardbprd.us.com/backup_20191114_144441.olr -

2019/11/25 12:47:06 CLSRSC-595: Executing upgrade step 12 of 12: 'InstallACFS'.

CRS-4123: Oracle High Availability Services has been started.

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'kardbprd.us.com'

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'kardbprd.us.com' has completed

CRS-4133: Oracle High Availability Services has been stopped.

CRS-4123: Oracle High Availability Services has been started.

2019/11/25 12:47:53 CLSRSC-327: Successfully configured Oracle Restart for a standalone server

Oracle DBA's

Tags

Sunday, December 8, 2019

Oracle Grid Upgrade 12.2.0.1 to 18.3.0.0

Upgrading Oracle Grid Infrastructure

Oracle Grid Upgrade 12.2.0.1 to 18c

Upgrading to Oracle Database 18c

Upgrade Grid Infrastructure to 18c

Oracle Grid upgrade from 12g to 18c

STARTING with Upgrade

As root user:

STEP 2 Database Upgrade:

Oracle database upgrade from 12g to 18c

physical standby without duplicate command

Search This Blog