ORACLE - DBA Tips Corner

Oracle DBA Tips Corner

Return to the Oracle DBA Tips Corner.

How to use Files in place of Real Disk Devices for ASM - (Linux)

by Jeff Hunter, Sr. Database Administrator

Contents

Overview
Creating Files for use by ASM
Create ASM Disk Groups
Startup Scripts

This article provides the steps to create and configure blank files (instead of real disk devices) for use in testing Oracle's Automatic Storage Manager (ASM) on the Linux platform. This is a handy trick that can be used when the DBA needs to test ASM on a machine that has no free disk partitions or no free disk devices available. Please note that this is intended for testing purposes only!

Creating Files for use by ASM

The first step is to identify an already partitioned and formatted hard disk that contains enough space to contain the blank files to be used as ASM disk devices.
For the purpose of this example, I have enough room on my local hard disk (/dev/sda1) to create four files at 100MB each. I want to create one disk group that contains four disks. The disk group will contain two failure groups and each failure group will be created using two disks.
From within the Linux O/S platform, perform the following actions:

Create Files using "dd"
# mkdir /asmdisks # chown oracle:dba /asmdisks # su - oracle $ dd if=/dev/zero of=/asmdisks/_file_disk1 bs=1k count=100000 $ dd if=/dev/zero of=/asmdisks/_file_disk2 bs=1k count=100000 $ dd if=/dev/zero of=/asmdisks/_file_disk3 bs=1k count=100000 $ dd if=/dev/zero of=/asmdisks/_file_disk4 bs=1k count=100000
We now have four files to be used as virtual disks of 100MB each:
$ ls -l /asmdisks total 400432 -rw-r--r-- 1 oracle dba 102400000 Jun 8 15:30 _file_disk1 -rw-r--r-- 1 oracle dba 102400000 Jun 8 15:31 _file_disk2 -rw-r--r-- 1 oracle dba 102400000 Jun 8 15:31 _file_disk3 -rw-r--r-- 1 oracle dba 102400000 Jun 8 15:31 _file_disk4

Associate Loop Device with the New Files
The next step is to use the losetup command to associate a loop device with a file. This needs to be performed as the root user account:
# losetup /dev/loop1 /asmdisks/_file_disk1 # losetup /dev/loop2 /asmdisks/_file_disk2 # losetup /dev/loop3 /asmdisks/_file_disk3 # losetup /dev/loop4 /asmdisks/_file_disk4

Bind RAW Devices to Existing Block Devices
The next critical step is to use the raw utility to bind a RAW device to each of the block devices we already created. Again, this needs to be performed as the root user account:
# raw /dev/raw/raw1 /dev/loop1 /dev/raw/raw1: bound to major 7, minor 1 # raw /dev/raw/raw2 /dev/loop2 /dev/raw/raw2: bound to major 7, minor 2 # raw /dev/raw/raw3 /dev/loop3 /dev/raw/raw3: bound to major 7, minor 3 # raw /dev/raw/raw4 /dev/loop4 /dev/raw/raw4: bound to major 7, minor 4

Change Ownership of RAW Devices
Finally, let's change the ownership of all four RAW devices:
# chown oracle:dba /dev/raw/raw1 # chown oracle:dba /dev/raw/raw2 # chown oracle:dba /dev/raw/raw3 # chown oracle:dba /dev/raw/raw4 # chmod 660 /dev/raw/raw1 # chmod 660 /dev/raw/raw2 # chmod 660 /dev/raw/raw3 # chmod 660 /dev/raw/raw4

After completeting the above steps, the O/S will see four free 'devices' now available for ASM!

Create ASM Disk Groups

Ok, so now that we have devices that can be seen by the O/S, we can now discover these disks within ASM and then create our ASM disk group.
For the purpose of this example, I already have an ASM instance running on the same node named "+ASM".

For a detailed article on configuring ASM, see my article: "Manually Creating an ASM Instance".

Let's start by determining if Oracle can find these four new disks:
# ls -l /dev/raw/raw[1234] crw-rw---- 1 oracle dba 162, 1 Jun 2 22:04 /dev/raw/raw1 crw-rw---- 1 oracle dba 162, 2 Jun 2 22:04 /dev/raw/raw2 crw-rw---- 1 oracle dba 162, 3 Jun 2 22:04 /dev/raw/raw3 crw-rw---- 1 oracle dba 162, 4 Jun 2 22:04 /dev/raw/raw4
The view V$ASM_DISK can be queried from the ASM instance to determine which disks are being used or may potentially be used as ASM disks. Note that you must log into the ASM instance with SYSDBA privileges. Here is the query that I ran from the ASM instance as the oracle user account:
$ ORACLE_SID=+ASM; export ORACLE_SID $ sqlplus "/ as sysdba" SQL> SELECT group_number, disk_number, mount_status, header_status, state, path 2 FROM v$asm_disk GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU STATE PATH ------------ ----------- ------- ------------ -------- --------------- 0 0 CLOSED CANDIDATE NORMAL /dev/raw/raw1 0 1 CLOSED CANDIDATE NORMAL /dev/raw/raw2 0 2 CLOSED CANDIDATE NORMAL /dev/raw/raw3 0 3 CLOSED CANDIDATE NORMAL /dev/raw/raw4
Note the value of zero in the GROUP_NUMBER column for all four disks. This indicates that a disk is available but hasn't yet been assigned to a disk group. The next section details the steps for creating a disk group.

Using SQL*Plus, the following will create a disk group with normal redundancy and two failure groups:
$ ORACLE_SID=+ASM; export ORACLE_SID $ sqlplus "/ as sysdba" SQL> CREATE DISKGROUP testdb_data1 NORMAL REDUNDANCY 2 FAILGROUP controller1 DISK '/dev/raw/raw1', '/dev/raw/raw2' 3 FAILGROUP controller2 DISK '/dev/raw/raw3', '/dev/raw/raw4'; Diskgroup created.

Now, let's take a look at the new disk group and disk details:
SQL> select group_number, name, total_mb, free_mb, state, type 2 from v$asm_diskgroup; GROUP_NUMBER NAME TOTAL_MB FREE_MB STATE TYPE ------------ -------------- ---------- ---------- ----------- ------ 1 TESTDB_DATA1 388 282 MOUNTED NORMAL SQL> select group_number, disk_number, mount_status, header_status, state, path, failgroup 2 from v$asm_disk; GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU STATE PATH FAILGROUP ------------ ----------- ------- ------------ -------- --------------- ------------ 1 0 CACHED MEMBER NORMAL /dev/raw/raw1 CONTROLLER1 1 1 CACHED MEMBER NORMAL /dev/raw/raw2 CONTROLLER1 1 2 CACHED MEMBER NORMAL /dev/raw/raw3 CONTROLLER2 1 3 CACHED MEMBER NORMAL /dev/raw/raw4 CONTROLLER2

Startup Scripts

Many of the above O/S commands will need to be put into a startup script that runs BEFORE the database starts. For my example, I use a file named /etc/init.d/dbora to start the database. I simply add the following command before starting the database:
... /sbin/losetup /dev/loop1 /asmdisks/_file_disk1; sleep 2 /sbin/losetup /dev/loop2 /asmdisks/_file_disk2; sleep 2 /sbin/losetup /dev/loop3 /asmdisks/_file_disk3; sleep 2 /sbin/losetup /dev/loop4 /asmdisks/_file_disk4; sleep 2 /usr/bin/raw /dev/raw/raw1 /dev/loop1; sleep 2 /usr/bin/raw /dev/raw/raw2 /dev/loop2; sleep 2 /usr/bin/raw /dev/raw/raw3 /dev/loop3; sleep 2 /usr/bin/raw /dev/raw/raw4 /dev/loop4; sleep 2 /bin/chown oracle:dba /dev/raw/raw1 /bin/chown oracle:dba /dev/raw/raw2 /bin/chown oracle:dba /dev/raw/raw3 /bin/chown oracle:dba /dev/raw/raw4 /bin/chmod 660 /dev/raw/raw1 /bin/chmod 660 /dev/raw/raw2 /bin/chmod 660 /dev/raw/raw3 /bin/chmod 660 /dev/raw/raw4 sleep 120 su - $ORACLE_OWNER -c "$ORACLE_HOME/bin/dbstart" su - $ORACLE_OWNER -c "lsnrctl start listener" ...

The next step is to edit the /etc/oratab file to allow the dbora script to automatically start and stop databases. Simply alter the final field in the +ASM and TESTDB entry from N to Y. NOTE: Ensure that the ASM instance is started BEFORE any databases that are making use of disk groups contained in it.
... +ASM:/u01/app/oracle/product/10.1.0/db_1:Y TESTDB:/u01/app/oracle/product/10.1.0/db_1:Y ...

The final step to manually edit the script /etc/inittab so that the entry to respawn init.cssd comes before running the runlevel 3.

Orignal /etc/inittab file:
(...) # System initialization. si::sysinit:/etc/rc.d/rc.sysinit l0:0:wait:/etc/rc.d/rc 0 l1:1:wait:/etc/rc.d/rc 1 l2:2:wait:/etc/rc.d/rc 2 l3:3:wait:/etc/rc.d/rc 3 l4:4:wait:/etc/rc.d/rc 4 l5:5:wait:/etc/rc.d/rc 5 l6:6:wait:/etc/rc.d/rc 6 (...) h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null

Modified /etc/inittab file:
(...) # System initialization. si::sysinit:/etc/rc.d/rc.sysinit l0:0:wait:/etc/rc.d/rc 0 l1:1:wait:/etc/rc.d/rc 1 l2:2:wait:/etc/rc.d/rc 2 h1:35:respawn:/etc/init.d/init.cssd run >/dev/null 2>&1 </dev/null l3:3:wait:/etc/rc.d/rc 3 l4:4:wait:/etc/rc.d/rc 4 l5:5:wait:/etc/rc.d/rc 5 l6:6:wait:/etc/rc.d/rc 6 (...)

All articles, scripts and material located at the Internet address of http://www.idevelopment.info is the copyright of Jeffrey M. Hunter and is protected under copyright laws of the United States. This document may not be hosted on any other site without my express, prior, written permission. Application to host any of the material elsewhere can be made by contacting me at jhunter@idevelopment.info.

I have made every effort and taken great care in making sure that the material included on my web site is technically accurate, but I disclaim any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on it. I will in no case be liable for any monetary damages arising from such loss, damage or destruction.

Last modified on
Monday, 25-Jul-2005 12:55:44 EDT
Page Count: 11263