Oracle RAC Clusterware Startup Sequence in detail

 

Once the Operating system starts and finish the boot scrap process it reads /etc/init.d file via the initialization daemon called init or init.d. 

The init tab file is the one it triggers oracle high availability service daemon.

1. When a node of an Oracle Clusterware cluster starts, OHASD is started by platform-specific means like init.d in Linux. 

   OHASD is the root for bringing up Oracle Clusterware. 

   OHASD has access to the OLR (Oracle Local Registry) stored on the local file system. 

   OLR provides needed data to complete OHASD initialization.

2. OHASD brings up GPNPD and CSSD ( Cluster synchronization Service Daemon ). 

   CSSD has access to the GPNP Profile stored on the local file system. 

   This profile contains the following vital bootstrap data:

a. ASM Diskgroup Discovery String

b. ASM SPFILE location (Diskgroup name)

c. Name of the ASM Diskgroup containing the Voting Files

3. The Voting Files locations on ASM Disks are accessed by CSSD with well-known pointers in the ASM Disk headers and CSSD is able to complete initialization and start or join an existing cluster.

4. OHASD starts an ASM instance and ASM can now operate with CSSD initialized and operating. 

   The ASM instance uses special code to locate the contents of the ASM SPFILE, assuming it is stored in a Diskgroup.

5. With an ASM instance operating and its Diskgroups mounted, access to Clusterware’s OCR is available to CRSD.

6. OHASD starts CRSD with access to the OCR in an ASM Diskgroup.

7. Clusterware completes initialization and brings up other services under its control.

==================================================

1. System Boot and Init Process

   • When the operating system completes the boot process, it reads the /etc/init.d scripts through the init or init.d daemon. These scripts manage service startups on Linux systems.
   • The inittab file configuration triggers the Oracle High Availability Services Daemon (OHASD). On Linux, this means that as part of the system startup, OHASD is one of the first Clusterware processes to start.

2. OHASD Initialization and OLR Access

   • OHASD is the root daemon for initializing Oracle Clusterware. It has privileged access to the Oracle Local Registry (OLR) stored on each node’s local file system.
   • OLR contains vital configuration data specific to the node, including information needed for initial Clusterware setup and node-specific details. This data enables OHASD to complete its initialization.

3. Starting GPNPD and CSSD Daemons

   • Once OHASD is operational, it starts the Grid Plug and Play Daemon (GPNPD) and Cluster Synchronization Services Daemon (CSSD).
   • CSSD has access to the GPNP Profile (also stored locally on each node), which includes essential bootstrap data such as:
     • ASM Diskgroup Discovery String: Information to discover ASM disks on startup.
     • ASM SPFILE Location: Specifies the diskgroup where the ASM SPFILE (Server Parameter File) is stored.
     • ASM Diskgroup Name for Voting Files: Identifies the ASM diskgroup that contains the Voting Disk(s), which are critical for cluster node coordination.

4. CSSD and Voting Files

   • CSSD reads the GPNP Profile to locate the Voting Files stored in ASM disk headers. Voting Files are used for node membership and are crucial for cluster health monitoring.
   • With these pointers, CSSD completes its initialization and either starts a new cluster or joins an existing one.

5. Starting ASM Instance and Diskgroup Mounting

   • With CSSD running, OHASD proceeds to start the ASM (Automatic Storage Management) instance.
   • ASM operates in coordination with CSSD to manage and access storage. It uses the SPFILE located in the specified diskgroup for initialization.
   • Once the ASM instance is active, it mounts the necessary ASM diskgroups, including the ones containing the Voting Files and OCR (Oracle Cluster Registry).

6. CRSD Initialization and OCR Access

   • With ASM and its diskgroups mounted, access to the Oracle Cluster Registry (OCR) is now available. OCR, which is typically stored in an ASM diskgroup, contains cluster-wide configuration data necessary for CRSD operations.
   • OHASD starts the Cluster Ready Services Daemon (CRSD), which relies on OCR for retrieving information about cluster resources, dependencies, and node roles.

7. Completion of Clusterware Initialization

• After CRSD starts, Oracle Clusterware completes the remainder of its initialization. CRSD then begins managing high-availability resources, such as database instances, listeners, and other services under Oracle Clusterware control.

Key Points on OLR, Voting Files, and Cluster Components

• OLR: Essential for node-specific configuration and available locally to OHASD. If OLR is lost or corrupted, OHASD may fail to start correctly on that node, and the node may need recovery or reconfiguration.

• Voting Files: These are central to node membership and fencing decisions, helping to avoid split-brain scenarios. CSSD relies on the Voting Files for its initialization and operation, while CRSD uses OCR for managing cluster resources.

• ASM Instance: Plays a dual role, storing both Voting Files and OCR if they’re ASM-managed. ASM mounts the required diskgroups to provide Clusterware components with access to these files.

• Cluster Resource Management: CRSD uses OCR data to start and monitor all Oracle RAC resources, completing the initialization sequence and ensuring high availability.

In summary, each component in the Oracle Clusterware startup process depends on previous components, creating a cascading initialization. If any critical part (like OLR, Voting Files, or ASM diskgroups) is missing or corrupted, it can impact the whole startup sequence, potentially leading to node isolation or cluster inaccessibility until recovery steps are taken.