Measure IOPs in Unix/Linux

When discuss about storage performance, we talk about IOPs. But how to measure IOPs in Unix/Linux system?

In hard disk specification we will have disk throughput in MB/s. 

It is easy to translate MB/s into IOPS and vice versa, we need to do a little math:

IOPS = (MBps Throughput / KB per IO) * 1024
Or
MBps = (IOPS * KB per IO) / 1024

In fact, servers in real world never run on a single direct attached disk, instead, a RAID
disk, NAS, SAN…etc. Therefore it is not easy to calculate the result.

I’ve found a tiny little utility called fio, fio is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 13 different types of I/O engines (sync, mmap, libaio, posixaio, SG v3, splice, null, network, syslet, guasi, solarisaio, and more), I/O priorities (for newer Linux kernels), rate I/O, forked or threaded jobs, and much more. It can work on block devices as well as files. fio accepts job descriptions in a simple-to-understand text format. Several example job files are included. fio displays all sorts of I/O performance information. Fio is in wide use in many places, for both bench-marking  QA, and verification purposes. It supports Linux, FreeBSD, NetBSD, OS X, OpenSolaris, AIX, HP-UX, and Windows.

Installation
Download fio package from freecode.com, unpack the bz2 file and make install.
I create my first stress test as below, detail parameters please refer to README file comes together with the source code.

# vi random-write-test.fio
[random-write]
rw=randwrite
size=50M
direct=1
directory=/home/IOPs
numjobs=10
group_reporting
name=random-write-direct
bs=8k
runtime=1000

Start my first test


# fio -output=fio.result -latency-log -bandwidth-log random-write-test.fio
# cat fio.result
random-write-direct: (g=0): rw=randwrite, bs=8K-8K/8K-8K/8K-8K, ioengine=sync, iodepth=1
...
random-write-direct: (g=0): rw=randwrite, bs=8K-8K/8K-8K/8K-8K, ioengine=sync, iodepth=1
fio-2.0.10
Starting 10 processes

random-write-direct: (groupid=0, jobs=10): err= 0: pid=9785: Fri Nov 16 14:46:15 2012
write: io=512000KB, bw=71759KB/s, iops=8969 , runt= 7135msec
clat (usec): min=330 , max=180178 , avg=1099.13, stdev=5571.55
lat (usec): min=331 , max=180179 , avg=1099.97, stdev=5571.57
clat percentiles (usec):
| 1.00th=[ 532], 5.00th=[ 604], 10.00th=[ 636], 20.00th=[ 660],
| 30.00th=[ 676], 40.00th=[ 692], 50.00th=[ 708], 60.00th=[ 724],
| 70.00th=[ 756], 80.00th=[ 788], 90.00th=[ 852], 95.00th=[ 940],
| 99.00th=[ 2928], 99.50th=[17024], 99.90th=[105984], 99.95th=[136192],
| 99.99th=[179200]
bw (KB/s) : min= 3056, max=10752, per=10.37%, avg=7443.05, stdev=2092.89
lat (usec) : 500=0.37%, 750=68.65%, 1000=27.01%
lat (msec) : 2=2.76%, 4=0.26%, 10=0.27%, 20=0.21%, 50=0.20%
lat (msec) : 100=0.17%, 250=0.10%
cpu : usr=0.82%, sys=2.28%, ctx=64171, majf=0, minf=259
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=64000/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
WRITE: io=512000KB, aggrb=71758KB/s, minb=71758KB/s, maxb=71758KB/s, mint=7135msec, maxt=7135msec

Disk stats (read/write): sda: ios=0/64233, merge=0/17, ticks=0/66923, in_queue=66925, util=98.46%

 

Result is 71759KB/s, remember my block size setting is 8KB? 

Therefore 71759KB /8 = 8969 IOP/s.
replica rolex rolex replica