How to Add Dropped Disks Back into Diskgroup

 

Adding Dropped Disks Back into Diskgroup

The steps documented in this post is helpful if you have dropped disks which is still hanging and ASM.repair_time parameter is already over. 

Further select * from v$asm_operations shows error.

Although the disk is still showing dropping but if you try to bring it online it fails. If you try to force drop it, it fails.

Step 1: Identify the missing/dropped disk for a particular disk group

select GROUP_NUMBER,NAME from  v$asm_diskgroup;

select name,header_status,mount_status,path from v$asm_disk where GROUP_NUMBER=2;

Step2: Find ASM disk level for each dropped disks

A> Scan ASM log file for this info

B> Check output of MD_BACKUP before the issue

In my case it was FRA_01 Diskgroup where one of storage site went down causing the issue. After scanning ASM alertlog  I found the following ASM disks which were part of the ASM Disk group but Currently these are the disks which is appearing as dropping.

 

failgroup FRA_01_DC01 disk
'ORCL:DC01_DB_CLU01_MPATHJ' name FRA_01_0004 size 51200M,
'ORCL:DC01_DB_CLU01_MPATHL' name FRA_01_0005 size 51200M,
'ORCL:DC01_DB_CLU01_MPATHM' name FRA_01_0006 size 51200M,
'ORCL:DC01_DB_CLU01_MPATHN' name FRA_01_0007 size 51200M

Step3: Ensure disk is physically present

Lets ensure that these disks are back after storage issue if fixed. You can only add them back if they are physically present

as root.

oracleasm querydisk -v -p DC01_DB_CLU01_MPATHJ
oracleasm querydisk -v -p DC01_DB_CLU01_MPATHL
oracleasm querydisk -v -p DC01_DB_CLU01_MPATHM
oracleasm querydisk -v -p DC01_DB_CLU01_MPATHN

Step4: Verify Disk Membership

Verify if that the identified disks are not part of any valid diskgroup and is not online
It could me member of an unknown diskgroup but then the mount status must be closed.

select name,header_status,mount_status,path from v$asm_disk where path in
('ORCL:DC01_DB_CLU01_MPATHJ',
 'ORCL:DC01_DB_CLU01_MPATHL',
 'ORCL:DC01_DB_CLU01_MPATHM',
 'ORCL:DC01_DB_CLU01_MPATHN');

SQL>   2    3    4    5
NAME                           HEADER_STATU MOUNT_S PATH
------------------------------ ------------ ------- ------------------------------
                               MEMBER       CLOSED  ORCL:DC01_DB_CLU01_MPATHJ
                               MEMBER       CLOSED  ORCL:DC01_DB_CLU01_MPATHN
                               MEMBER       CLOSED  ORCL:DC01_DB_CLU01_MPATHM
                               MEMBER       CLOSED  ORCL:DC01_DB_CLU01_MPATHL

Step5:Force Add the disks Back to the diskgroup.

Ensure to use proper disk name , fail group name and rebalance power. In my case I was only able to add one disk at a time. If tried adding multiple got issue like Disk is already part/member of diskgroup.

alter diskgroup FRA_01 add failgroup FRA_01_DC01 disk
'ORCL:DC01_DB_CLU01_MPATHL' name FRA_01_0005
force rebalance power 11;

alter diskgroup FRA_01 add failgroup FRA_01_DC01 disk
'ORCL:DC01_DB_CLU01_MPATHM' name FRA_01_0006
force rebalance power 11;

...............................

...............................

................................

Step6:Wait for rebalance to complete.

select * from gv$asm_operation;

Once rebalance operation is done you will find the disks which were hanging in dropping state are already gone and disk group is not healthy all the disks are back to online.

Step 7: Verify

SQL> select GROUP_NUMBER,NAME from  v$asm_diskgroup;

SQL> select name,header_status,mount_status,path from v$asm_disk where GROUP_NUMBER=2;

SQL> select name,header_status,mount_status,path from v$asm_disk;