11.2.0.4 RAC数据库存储迁移

OS: Solaris 10 SPARC 64bit
GI: 11.2.0.4.2
DB: 11.2.0.4.2
OCR存储方式: ASM
DB存储方式: ASM

 1， 新的存储设备划分LUN并且做映射后，在OS通过"format"可以看到新的磁盘，修改磁盘属主及权限；

chown grid:asmadmin /dev/rdsk/c9t600601601DC0310064D1C64F2FDBE311d0s6
chown grid:asmadmin /dev/rdsk/c9t600601601DC0310066D1C64F2FDBE311d0s6
chown grid:asmadmin /dev/rdsk/c9t600601601DC0310068D1C64F2FDBE311d0s6
chown grid:asmadmin /dev/rdsk/c9t600601601DC031006AD1C64F2FDBE311d0s6
chown grid:asmadmin /dev/rdsk/c9t600601601DC031006CD1C64F2FDBE311d0s6
chown grid:asmadmin /dev/rdsk/c9t600601601DC031006ED1C64F2FDBE311d0s6
chown grid:asmadmin /dev/rdsk/c9t600601601DC0310070D1C64F2FDBE311d0s6
chown grid:asmadmin /dev/rdsk/c9t600601601DC03100C2A66C942FDBE311d0s6
chown grid:asmadmin /dev/rdsk/c9t600601601DC03100C4A66C942FDBE311d0s6
chown grid:asmadmin /dev/rdsk/c9t600601601DC03100C6A66C942FDBE311d0s6

chmod 660 /dev/rdsk/c9t600601601DC0310064D1C64F2FDBE311d0s6
chmod 660 /dev/rdsk/c9t600601601DC0310066D1C64F2FDBE311d0s6
chmod 660 /dev/rdsk/c9t600601601DC0310068D1C64F2FDBE311d0s6
chmod 660 /dev/rdsk/c9t600601601DC031006AD1C64F2FDBE311d0s6
chmod 660 /dev/rdsk/c9t600601601DC031006CD1C64F2FDBE311d0s6
chmod 660 /dev/rdsk/c9t600601601DC031006ED1C64F2FDBE311d0s6
chmod 660 /dev/rdsk/c9t600601601DC0310070D1C64F2FDBE311d0s6
chmod 660 /dev/rdsk/c9t600601601DC03100C2A66C942FDBE311d0s6
chmod 660 /dev/rdsk/c9t600601601DC03100C4A66C942FDBE311d0s6
chmod 660 /dev/rdsk/c9t600601601DC03100C6A66C942FDBE311d0s6

2，通过ASM磁盘的Rebalance迁移+DATA磁盘组，每添加或删除一块磁盘后查看v$asm_disk，检查磁盘的MOUNT_STATUS和HEADER_STATUS，以及v$asm_operation来确认Rebalance已经完成，再进行下一项操作；

SQL> ALTER DISKGROUP data DROP DISK 'DATA_0000' ADD DISK '/dev/rdsk/c9t600601601DC0310064D1C64F2FDBE311d0s6' REBALANCE POWER 5;

Diskgroup altered.

SQL> ALTER DISKGROUP data DROP DISK 'DATA_0001' ADD DISK '/dev/rdsk/c9t600601601DC0310066D1C64F2FDBE311d0s6' REBALANCE POWER 5;

Diskgroup altered.

SQL> ALTER DISKGROUP data DROP DISK 'DATA_0002' ADD DISK '/dev/rdsk/c9t600601601DC0310068D1C64F2FDBE311d0s6' REBALANCE POWER 5;

Diskgroup altered.

SQL> ALTER DISKGROUP data DROP DISK 'DATA_0003' ADD DISK '/dev/rdsk/c9t600601601DC031006AD1C64F2FDBE311d0s6' REBALANCE POWER 5;

Diskgroup altered.

SQL> ALTER DISKGROUP data DROP DISK 'DATA_0004' ADD DISK '/dev/rdsk/c9t600601601DC031006CD1C64F2FDBE311d0s6' REBALANCE POWER 5;

Diskgroup altered.

3，迁移+FRA磁盘组

SQL> ALTER DISKGROUP fra DROP DISK 'FRA_0000' ADD DISK '/dev/rdsk/c9t600601601DC031006ED1C64F2FDBE311d0s6' REBALANCE POWER 5;

Diskgroup altered.

SQL> ALTER DISKGROUP fra DROP DISK 'FRA_0001' ADD DISK '/dev/rdsk/c9t600601601DC0310070D1C64F2FDBE311d0s6' REBALANCE POWER 5;

Diskgroup altered.

4，迁移OCR；
    在集群其中一个节点手动备份OCR

-bash-3.2# pwd
/u01/app/11.2.0/grid/bin
-bash-3.2# ./ocrconfig -manualbackup

nnmdb-node01     2014/05/16 12:59:12     /u01/app/11.2.0/grid/cdata/nnmdb-cluster/backup_20140516_125912.ocr
-bash-3.2# ./ocrconfig -showbackup manual

nnmdb-node01     2014/05/16 12:59:12     /u01/app/11.2.0/grid/cdata/nnmdb-cluster/backup_20140516_125912.ocr

     在其一个节点使用新磁盘创建OCR磁盘组+OCRV

SQL> CREATE DISKGROUP OCRV NORMAL REDUNDANCY
  2  FAILGROUP ocrv_0000 DISK '/dev/rdsk/c9t600601601DC03100C2A66C942FDBE311d0s6'
  3  FAILGROUP ocrv_0001 DISK '/dev/rdsk/c9t600601601DC03100C4A66C942FDBE311d0s6'
  4  QUORUM FAILGROUP ocrv_0002 DISK '/dev/rdsk/c9t600601601DC03100C6A66C942FDBE311d0s6'
  5  ATTRIBUTE 'COMPATIBLE.ASM'='11.2.0.0.0';

Diskgroup created.

     在另外一个节点的ASM实例手动挂载+OCRV磁盘组，确保+OCRV在所有节点都是MOUNT状态

SQL> ALTER DISKGROUP ocrv MOUNT;

Diskgroup altered.

     迁移OCR文件，在其中一个节点执行

-bash-3.2# ./ocrconfig -add +OCRV
-bash-3.2# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version : 3
         Total space (kbytes) : 262120
         Used space (kbytes) : 3512
         Available space (kbytes) : 258608
         ID : 526301015
         Device/File Name : +OCR
                                    Device/File integrity check succeeded
         Device/File Name : +OCRV
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

-bash-3.2# ./ocrconfig -delete +OCR
-bash-3.2# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version : 3
         Total space (kbytes) : 262120
         Used space (kbytes) : 3512
         Available space (kbytes) : 258608
         ID : 526301015
         Device/File Name : +OCRV
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

-bash-3.2#

     注意：如果现有的环境只存在一个OCR文件，不支持“ocrconfig -replace ... -replacement ...”命令替换老的OCR文件

-bash-3.2# ./ocrconfig -replace '+OCR' -replacement '+OCRV'
PROT-28: Cannot delete or replace the only configured Oracle Cluster Registry location

5， 迁移Vote disk

-bash-3.2# ./crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
-- ----- ----------------- --------- ---------
 1. ONLINE   3a480248ccbd4fafbf9cd195101034df (/dev/rdsk/c9t6006016042303500B467DF7789C3E311d0s6) [OCR]
 2. ONLINE   85a593e0a5d54fedbf627cbb3aa29f03 (/dev/rdsk/c9t6006016042303500B667DF7789C3E311d0s6) [OCR]
 3. ONLINE   920777d8795b4f58bfff0b6e0b0017e8 (/dev/rdsk/c9t6006016042303500B867DF7789C3E311d0s6) [OCR]
Located 3 voting disk(s).
-bash-3.2#
-bash-3.2# ./crsctl replace votedisk +OCRV
Successful addition of voting disk 242611d97d8b4fb4bf3e1739476e157b.
Successful addition of voting disk a2bc139413fc4f45bf3c4fb76763d03d.
Successful addition of voting disk 2abe81f543ab4fdbbff70a50326f0cec.
Successful deletion of voting disk 3a480248ccbd4fafbf9cd195101034df.
Successful deletion of voting disk 85a593e0a5d54fedbf627cbb3aa29f03.
Successful deletion of voting disk 920777d8795b4f58bfff0b6e0b0017e8.
Successfully replaced voting disk group with +OCRV.
CRS-4266: Voting file(s) successfully replaced
-bash-3.2#
-bash-3.2# ./crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
-- ----- ----------------- --------- ---------
 1. ONLINE   242611d97d8b4fb4bf3e1739476e157b (/dev/rdsk/c9t600601601DC03100C6A66C942FDBE311d0s6) [OCRV]
 2. ONLINE   a2bc139413fc4f45bf3c4fb76763d03d (/dev/rdsk/c9t600601601DC03100C2A66C942FDBE311d0s6) [OCRV]
 3. ONLINE   2abe81f543ab4fdbbff70a50326f0cec (/dev/rdsk/c9t600601601DC03100C4A66C942FDBE311d0s6) [OCRV]
Located 3 voting disk(s).

6，迁移ASM实例SPFILE
     在其中一个节点ASM实例的+OCRV磁盘组上创建SPFILE，重启各节点的ASM实例

SQL> create pfile='/tmp/pfile' from spfile;

File created.
SQL> !more /tmp/pfile
+ASM1.asm_diskgroups='DATA','FRA'#Manual Mount
+ASM2.asm_diskgroups='DATA','FRA'#Manual Mount
*.asm_power_limit=1
*.diagnostic_dest='/u01/app/grid'
*.instance_type='asm'
*.large_pool_size=12M
*.remote_login_passwordfile='EXCLUSIVE'

SQL> create spfile='+OCRV' from pfile='/tmp/pfile';

File created.

7，删除旧的OCR磁盘组
     将+OCR磁盘组从集群资源中移除

-bash-3.2# ./srvctl stop diskgroup -g ocr -n nnmdb-node01
-bash-3.2# ./srvctl stop diskgroup -g ocr -n nnmdb-node02
-bash-3.2# ./srvctl remove diskgroup -g ocr -f

     在两个节点的ASM实例分别DISMOUNT磁盘组

SQL> ALTER DISKGROUP ocr DISMOUNT FORCE;

Diskgroup altered.

     在其中一个节点删除磁盘组

SQL> DROP DISKGROUP ocr FORCE INCLUDING CONTENTS;

Diskgroup dropped.

     存储迁移全部完成，旧的磁盘/LUN可以从服务器物理移除。