Oracle DBA Tips Corner |
How to use Files in place of Real Disk Devices for ASM - (Linux)
by Jeff Hunter, Sr. Database Administrator
Contents
Overview
Creating Files for use by ASM
For the purpose of this example,
I have enough room on my local hard disk (/dev/sda1) to
create four files at 100MB each. I want to create one disk group
that contains four disks. The disk group will contain two failure groups
and each failure group will be created using two disks.
From within the Linux O/S platform,
perform the following actions:
The next step is to use the losetup command to associate
a loop device with a file. This needs to be performed as the root
user account:
The next critical step is to use the raw utility to bind a RAW device
to each of the block devices we already created. Again, this needs to be performed as
the root user account:
Finally, let's change the ownership of all four RAW devices:
After completeting the above steps, the O/S will see four free 'devices'
now available for ASM!
Create ASM Disk Groups
For the purpose of this example, I already have an ASM instance running
on the same node named "+ASM".
Let's start by determining if Oracle can find these four new disks:
Using SQL*Plus, the following will create a disk group with normal redundancy
and two failure groups:
Now, let's take a look at the new disk group and disk details:
Startup Scripts
The next step is to edit the /etc/oratab
file to allow the dbora script to automatically start and stop
databases. Simply alter the final field in the +ASM and TESTDB
entry from N to Y. NOTE: Ensure
that the ASM instance is started BEFORE any databases that are
making use of disk groups contained in it.
The final step to manually edit the script /etc/inittab so
that the entry to respawn init.cssd comes before
running the runlevel 3.
All articles, scripts and material located at the Internet address of http://www.idevelopment.info is the copyright of Jeffrey M. Hunter
and is protected under copyright laws of the United States. This document may not be hosted on any other site without my express,
prior, written permission. Application to host any of the material elsewhere can be made by contacting me at jhunter@idevelopment.info.
I have made every effort and taken great care in making sure that the material included on my web site is technically accurate,
but I disclaim any and all responsibility for any loss, damage or destruction of data or any other property which may arise from
relying on it. I will in no case be liable for any monetary damages arising from such loss, damage or destruction.
This article provides the steps to create and configure
blank files (instead of real disk devices) for use
in testing Oracle's Automatic Storage Manager (ASM) on the
Linux platform. This is a
handy trick that can be used when the DBA needs to test
ASM on a machine that has no free disk partitions or no
free disk devices available. Please note that this is intended
for testing purposes only!
The first step is to identify an already partitioned and formatted
hard disk that contains enough space to contain the blank files
to be used as ASM disk devices.
# mkdir /asmdisks
# chown oracle:dba /asmdisks
# su - oracle
$ dd if=/dev/zero of=/asmdisks/_file_disk1 bs=1k count=100000
$ dd if=/dev/zero of=/asmdisks/_file_disk2 bs=1k count=100000
$ dd if=/dev/zero of=/asmdisks/_file_disk3 bs=1k count=100000
$ dd if=/dev/zero of=/asmdisks/_file_disk4 bs=1k count=100000
We now have four files to be used as virtual disks of 100MB each:
$ ls -l /asmdisks
total 400432
-rw-r--r-- 1 oracle dba 102400000 Jun 8 15:30 _file_disk1
-rw-r--r-- 1 oracle dba 102400000 Jun 8 15:31 _file_disk2
-rw-r--r-- 1 oracle dba 102400000 Jun 8 15:31 _file_disk3
-rw-r--r-- 1 oracle dba 102400000 Jun 8 15:31 _file_disk4
# losetup /dev/loop1 /asmdisks/_file_disk1
# losetup /dev/loop2 /asmdisks/_file_disk2
# losetup /dev/loop3 /asmdisks/_file_disk3
# losetup /dev/loop4 /asmdisks/_file_disk4
# raw /dev/raw/raw1 /dev/loop1
/dev/raw/raw1: bound to major 7, minor 1
# raw /dev/raw/raw2 /dev/loop2
/dev/raw/raw2: bound to major 7, minor 2
# raw /dev/raw/raw3 /dev/loop3
/dev/raw/raw3: bound to major 7, minor 3
# raw /dev/raw/raw4 /dev/loop4
/dev/raw/raw4: bound to major 7, minor 4
# chown oracle:dba /dev/raw/raw1
# chown oracle:dba /dev/raw/raw2
# chown oracle:dba /dev/raw/raw3
# chown oracle:dba /dev/raw/raw4
# chmod 660 /dev/raw/raw1
# chmod 660 /dev/raw/raw2
# chmod 660 /dev/raw/raw3
# chmod 660 /dev/raw/raw4
Ok, so now that we have devices that can be seen by the O/S, we can
now discover these disks within ASM and then create our ASM disk group.
For a detailed article on configuring ASM, see my article:
"Manually Creating an ASM Instance".
# ls -l /dev/raw/raw[1234]
crw-rw---- 1 oracle dba 162, 1 Jun 2 22:04 /dev/raw/raw1
crw-rw---- 1 oracle dba 162, 2 Jun 2 22:04 /dev/raw/raw2
crw-rw---- 1 oracle dba 162, 3 Jun 2 22:04 /dev/raw/raw3
crw-rw---- 1 oracle dba 162, 4 Jun 2 22:04 /dev/raw/raw4
The view
V$ASM_DISK can be queried from the ASM instance to determine
which disks are being used or may potentially be used as ASM disks. Note
that you must log into the ASM instance with SYSDBA privileges.
Here is the query that I ran from the ASM instance as the oracle
user account:
$ ORACLE_SID=+ASM; export ORACLE_SID
$ sqlplus "/ as sysdba"
SQL> SELECT group_number, disk_number, mount_status, header_status, state, path
2 FROM v$asm_disk
GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU STATE PATH
------------ ----------- ------- ------------ -------- ---------------
0 0 CLOSED CANDIDATE NORMAL /dev/raw/raw1
0 1 CLOSED CANDIDATE NORMAL /dev/raw/raw2
0 2 CLOSED CANDIDATE NORMAL /dev/raw/raw3
0 3 CLOSED CANDIDATE NORMAL /dev/raw/raw4
Note the value of zero in the GROUP_NUMBER column for all four disks. This indicates
that a disk is available but hasn't yet been assigned to a disk group. The next
section details the steps for creating a disk group.
$ ORACLE_SID=+ASM; export ORACLE_SID
$ sqlplus "/ as sysdba"
SQL> CREATE DISKGROUP testdb_data1 NORMAL REDUNDANCY
2 FAILGROUP controller1 DISK '/dev/raw/raw1', '/dev/raw/raw2'
3 FAILGROUP controller2 DISK '/dev/raw/raw3', '/dev/raw/raw4';
Diskgroup created.
SQL> select group_number, name, total_mb, free_mb, state, type
2 from v$asm_diskgroup;
GROUP_NUMBER NAME TOTAL_MB FREE_MB STATE TYPE
------------ -------------- ---------- ---------- ----------- ------
1 TESTDB_DATA1 388 282 MOUNTED NORMAL
SQL> select group_number, disk_number, mount_status, header_status, state, path, failgroup
2 from v$asm_disk;
GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU STATE PATH FAILGROUP
------------ ----------- ------- ------------ -------- --------------- ------------
1 0 CACHED MEMBER NORMAL /dev/raw/raw1 CONTROLLER1
1 1 CACHED MEMBER NORMAL /dev/raw/raw2 CONTROLLER1
1 2 CACHED MEMBER NORMAL /dev/raw/raw3 CONTROLLER2
1 3 CACHED MEMBER NORMAL /dev/raw/raw4 CONTROLLER2
Many of the above O/S commands will need to be put into a startup
script that runs BEFORE the database starts. For my example, I use
a file named /etc/init.d/dbora to start the database. I simply
add the following command before starting the database:
...
/sbin/losetup /dev/loop1 /asmdisks/_file_disk1; sleep 2
/sbin/losetup /dev/loop2 /asmdisks/_file_disk2; sleep 2
/sbin/losetup /dev/loop3 /asmdisks/_file_disk3; sleep 2
/sbin/losetup /dev/loop4 /asmdisks/_file_disk4; sleep 2
/usr/bin/raw /dev/raw/raw1 /dev/loop1; sleep 2
/usr/bin/raw /dev/raw/raw2 /dev/loop2; sleep 2
/usr/bin/raw /dev/raw/raw3 /dev/loop3; sleep 2
/usr/bin/raw /dev/raw/raw4 /dev/loop4; sleep 2
/bin/chown oracle:dba /dev/raw/raw1
/bin/chown oracle:dba /dev/raw/raw2
/bin/chown oracle:dba /dev/raw/raw3
/bin/chown oracle:dba /dev/raw/raw4
/bin/chmod 660 /dev/raw/raw1
/bin/chmod 660 /dev/raw/raw2
/bin/chmod 660 /dev/raw/raw3
/bin/chmod 660 /dev/raw/raw4
sleep 120
su - $ORACLE_OWNER -c "$ORACLE_HOME/bin/dbstart"
su - $ORACLE_OWNER -c "lsnrctl start listener"
...
...
+ASM:/u01/app/oracle/product/10.1.0/db_1:Y
TESTDB:/u01/app/oracle/product/10.1.0/db_1:Y
...
(...)
# System initialization.
si::sysinit:/etc/rc.d/rc.sysinit
l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6
(...)
h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null
(...)
# System initialization.
si::sysinit:/etc/rc.d/rc.sysinit
l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6
(...)
Monday, 25-Jul-2005 12:55:44 EDT
Page Count: 11263