DBA Tips Archive for Oracle

How to use Files in place of Real Disk Devices for ASM - (10g, Linux)

by Jeff Hunter, Sr. Database Administrator

Contents

• Introduction
• Create Device Files for use by ASM
• Create ASM Disk Group
• Startup Scripts
• About the Author

Introduction

This guide provides the steps to create and configure blank files (instead of real disk devices) for use in testing Oracle's Automatic Storage Manager (Oracle ASM) on the Linux platform. This is a handy trick that can be used when the DBA needs to test Oracle ASM on a machine that has no free disk partitions or no free disk devices available.

Create Device Files for use by ASM

The first step is to identify an already partitioned and formatted hard disk that contains enough space to store the blank files to be used as ASM disk devices.

A total of four (4) files at 100MB each will be created on the local hard disk (/dev/sda1). A single disk group will be created containing the four disks. The disk group will contain two failure groups and each failure group will be created using two disks.

From within the Linux O/S platform, perform the following actions.

1. Create new text files using "dd".

   # mkdir /asmdisks # chown oracle:dba /asmdisks # su - oracle $ dd if=/dev/zero of=/asmdisks/_file_disk1 bs=1k count=100000 $ dd if=/dev/zero of=/asmdisks/_file_disk2 bs=1k count=100000 $ dd if=/dev/zero of=/asmdisks/_file_disk3 bs=1k count=100000 $ dd if=/dev/zero of=/asmdisks/_file_disk4 bs=1k count=100000

   We now have four files to be used as virtual disks of 100MB each.

   $ ls -l /asmdisks total 400432 -rw-r--r-- 1 oracle dba 102400000 Jun 8 15:30 _file_disk1 -rw-r--r-- 1 oracle dba 102400000 Jun 8 15:31 _file_disk2 -rw-r--r-- 1 oracle dba 102400000 Jun 8 15:31 _file_disk3 -rw-r--r-- 1 oracle dba 102400000 Jun 8 15:31 _file_disk4

2. Use the losetup command to associate a loop device with a file. This needs to be performed as the root user account.

   # losetup /dev/loop1 /asmdisks/_file_disk1 # losetup /dev/loop2 /asmdisks/_file_disk2 # losetup /dev/loop3 /asmdisks/_file_disk3 # losetup /dev/loop4 /asmdisks/_file_disk4

3. Use the raw utility to bind a RAW device to each of the block devices we already created. Again, this needs to be performed as the root user account.

   # raw /dev/raw/raw1 /dev/loop1 /dev/raw/raw1: bound to major 7, minor 1 # raw /dev/raw/raw2 /dev/loop2 /dev/raw/raw2: bound to major 7, minor 2 # raw /dev/raw/raw3 /dev/loop3 /dev/raw/raw3: bound to major 7, minor 3 # raw /dev/raw/raw4 /dev/loop4 /dev/raw/raw4: bound to major 7, minor 4

4. Change the ownership of all four RAW devices.

   # chown oracle:dba /dev/raw/raw1 # chown oracle:dba /dev/raw/raw2 # chown oracle:dba /dev/raw/raw3 # chown oracle:dba /dev/raw/raw4 # chmod 660 /dev/raw/raw1 # chmod 660 /dev/raw/raw2 # chmod 660 /dev/raw/raw3 # chmod 660 /dev/raw/raw4

After completeting the above steps, the O/S will see four free 'devices' now available for Oracle ASM.

Create ASM Disk Group

For the purpose of this example, an Oracle ASM instance is already running on the node named +ASM.

With the new device files in place and seen by the O/S, discover these disks within Oracle ASM and then create the disk group.

Start by determining if Oracle can find these four new disks.

# ls -l /dev/raw/raw[1234] crw-rw---- 1 oracle dba 162, 1 Jun 2 22:04 /dev/raw/raw1 crw-rw---- 1 oracle dba 162, 2 Jun 2 22:04 /dev/raw/raw2 crw-rw---- 1 oracle dba 162, 3 Jun 2 22:04 /dev/raw/raw3 crw-rw---- 1 oracle dba 162, 4 Jun 2 22:04 /dev/raw/raw4

The view V$ASM_DISK can be queried from the ASM instance to determine which disks are being used or may potentially be used as ASM disks. Note that you must log into the Oracle ASM instance with SYSDBA privileges. Run the following query from the Oracle ASM instance as SYSDBA.

$ ORACLE_SID=+ASM; export ORACLE_SID $ sqlplus "/ as sysdba" SQL> SELECT group_number, disk_number, mount_status, header_status, state, path 2 FROM v$asm_disk GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU STATE PATH ------------ ----------- ------- ------------ -------- --------------- 0 0 CLOSED CANDIDATE NORMAL /dev/raw/raw1 0 1 CLOSED CANDIDATE NORMAL /dev/raw/raw2 0 2 CLOSED CANDIDATE NORMAL /dev/raw/raw3 0 3 CLOSED CANDIDATE NORMAL /dev/raw/raw4

Note the value of zero in the GROUP_NUMBER column for all four disks. This indicates that a disk is available but hasn't yet been assigned to a disk group.

Create a disk group with normal redundancy and two failure groups.

SQL> CREATE DISKGROUP testdb_data1 NORMAL REDUNDANCY 2 FAILGROUP controller1 DISK '/dev/raw/raw1', '/dev/raw/raw2' 3 FAILGROUP controller2 DISK '/dev/raw/raw3', '/dev/raw/raw4'; Diskgroup created.

Validate the new disk group and disk details.

SQL> select group_number, name, total_mb, free_mb, state, type 2 from v$asm_diskgroup; GROUP_NUMBER NAME TOTAL_MB FREE_MB STATE TYPE ------------ -------------- ---------- ---------- ----------- ------ 1 TESTDB_DATA1 388 282 MOUNTED NORMAL SQL> select group_number, disk_number, mount_status, header_status, state, path, failgroup 2 from v$asm_disk; GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU STATE PATH FAILGROUP ------------ ----------- ------- ------------ -------- --------------- ------------ 1 0 CACHED MEMBER NORMAL /dev/raw/raw1 CONTROLLER1 1 1 CACHED MEMBER NORMAL /dev/raw/raw2 CONTROLLER1 1 2 CACHED MEMBER NORMAL /dev/raw/raw3 CONTROLLER2 1 3 CACHED MEMBER NORMAL /dev/raw/raw4 CONTROLLER2

Startup Scripts

Many of the above O/S commands will need to be put into a startup script that runs before the database starts. For this example, a file named /etc/init.d/dbora is already in place to start the database during the boot process. Add the following commands before starting the database.

... /sbin/losetup /dev/loop1 /asmdisks/_file_disk1; sleep 2 /sbin/losetup /dev/loop2 /asmdisks/_file_disk2; sleep 2 /sbin/losetup /dev/loop3 /asmdisks/_file_disk3; sleep 2 /sbin/losetup /dev/loop4 /asmdisks/_file_disk4; sleep 2 /usr/bin/raw /dev/raw/raw1 /dev/loop1; sleep 2 /usr/bin/raw /dev/raw/raw2 /dev/loop2; sleep 2 /usr/bin/raw /dev/raw/raw3 /dev/loop3; sleep 2 /usr/bin/raw /dev/raw/raw4 /dev/loop4; sleep 2 /bin/chown oracle:dba /dev/raw/raw1 /bin/chown oracle:dba /dev/raw/raw2 /bin/chown oracle:dba /dev/raw/raw3 /bin/chown oracle:dba /dev/raw/raw4 /bin/chmod 660 /dev/raw/raw1 /bin/chmod 660 /dev/raw/raw2 /bin/chmod 660 /dev/raw/raw3 /bin/chmod 660 /dev/raw/raw4 sleep 120 su - $ORACLE_OWNER -c "$ORACLE_HOME/bin/dbstart" su - $ORACLE_OWNER -c "lsnrctl start listener" ...

The next step is to edit the /etc/oratab file to allow the dbora script to automatically start and stop databases. Alter the final field in the +ASM and TESTDB entry from N to Y. Ensure that the Oracle ASM instance is started before any databases that are making use of disk groups contained in it.

... +ASM:/u01/app/oracle/product/10.1.0/db_1:Y TESTDB:/u01/app/oracle/product/10.1.0/db_1:Y ...

The final step to manually edit the script /etc/inittab so that the entry to respawn init.cssd comes before running the runlevel 3.

Orignal /etc/inittab file.

(...) # System initialization. si::sysinit:/etc/rc.d/rc.sysinit l0:0:wait:/etc/rc.d/rc 0 l1:1:wait:/etc/rc.d/rc 1 l2:2:wait:/etc/rc.d/rc 2 l3:3:wait:/etc/rc.d/rc 3 l4:4:wait:/etc/rc.d/rc 4 l5:5:wait:/etc/rc.d/rc 5 l6:6:wait:/etc/rc.d/rc 6 (...) h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null

Modified /etc/inittab file.

(...) # System initialization. si::sysinit:/etc/rc.d/rc.sysinit l0:0:wait:/etc/rc.d/rc 0 l1:1:wait:/etc/rc.d/rc 1 l2:2:wait:/etc/rc.d/rc 2 h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null l3:3:wait:/etc/rc.d/rc 3 l4:4:wait:/etc/rc.d/rc 4 l5:5:wait:/etc/rc.d/rc 5 l6:6:wait:/etc/rc.d/rc 6 (...)

About the Author

Jeffrey Hunter is an Oracle Certified Professional, Java Development Certified Professional, Author, and an Oracle ACE. Jeff currently works as a Senior Database Administrator for The DBA Zone, Inc. located in Pittsburgh, Pennsylvania. His work includes advanced performance tuning, Java and PL/SQL programming, developing high availability solutions, capacity planning, database security, and physical / logical database design in a UNIX, Linux, and Windows server environment. Jeff's other interests include mathematical encryption theory, programming language processors (compilers and interpreters) in Java and C, LDAP, writing web-based database administration tools, and of course Linux. He has been a Sr. Database Administrator and Software Engineer for over 18 years and maintains his own website site at: http://www.iDevelopment.info. Jeff graduated from Stanislaus State University in Turlock, California, with a Bachelor's degree in Computer Science.