fio on Intel SSD DC P3608 (3.2TB)

# lspci
00:00.0 Host bridge: Intel Corporation Sky Lake Host Bridge/DRAM Registers (rev 07)
00:01.0 PCI bridge: Intel Corporation Sky Lake PCIe Controller (x16) (rev 07)
00:02.0 VGA compatible controller: Intel Corporation Sky Lake Integrated Graphics (rev 06)
00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller (rev 31)
00:14.2 Signal processing controller: Intel Corporation Sunrise Point-H Thermal subsystem (rev 31)
00:16.0 Communication controller: Intel Corporation Sunrise Point-H CSME HECI #1 (rev 31)
00:17.0 SATA controller: Intel Corporation Device a102 (rev 31)
00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #5 (rev f1)
00:1d.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #9 (rev f1)
00:1f.0 ISA bridge: Intel Corporation Sunrise Point-H LPC Controller (rev 31)
00:1f.2 Memory controller: Intel Corporation Sunrise Point-H PMC (rev 31)
00:1f.3 Audio device: Intel Corporation Sunrise Point-H HD Audio (rev 31)
00:1f.4 SMBus: Intel Corporation Sunrise Point-H SMBus (rev 31)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-V (rev 31)
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8718 16-Lane, 5-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ab)
02:01.0 PCI bridge: PLX Technology, Inc. PEX 8718 16-Lane, 5-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ab)
02:02.0 PCI bridge: PLX Technology, Inc. PEX 8718 16-Lane, 5-Port PCI Express Gen 3 (8.0 GT/s) Switch (rev ab)
03:00.0 Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 02)
04:00.0 Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 02)

# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    1
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 94
Model name:            Intel(R) Core(TM) i5-6400 CPU @ 2.70GHz
Stepping:              3
CPU MHz:               3300.328
BogoMIPS:              5422.27
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              6144K
NUMA node0 CPU(s):     0-3

# uname -r
3.10.0-327.4.4.el7.x86_64

# fio -rw=randwrite -bs=4k -size=100m -directory=/mnt/ -direct=1 -numjobs=50 -name=file1 -ioengine=libaio -group_reporting
file1: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
...
fio-2.2.8
Starting 50 processes
Jobs: 50 (f=50): [w(50)] [100.0% done] [0KB/829.9MB/0KB /s] [0/212K/0 iops] [eta 00m:00s]
file1: (groupid=0, jobs=50): err= 0: pid=4104: Mon Jan 25 22:53:49 2016
  write: io=5000.0MB, bw=971906KB/s, iops=242976, runt=  5268msec
    slat (usec): min=1, max=110, avg= 4.69, stdev= 4.58
    clat (usec): min=0, max=8200, avg=199.08, stdev=335.88
     lat (usec): min=10, max=8203, avg=203.83, stdev=335.92
    clat percentiles (usec):
     |  1.00th=[   11],  5.00th=[   14], 10.00th=[   19], 20.00th=[   30],
     | 30.00th=[   52], 40.00th=[   92], 50.00th=[  131], 60.00th=[  163],
     | 70.00th=[  191], 80.00th=[  227], 90.00th=[  318], 95.00th=[  668],
     | 99.00th=[ 1800], 99.50th=[ 2040], 99.90th=[ 2544], 99.95th=[ 3280],
     | 99.99th=[ 5856]
    bw (KB  /s): min=15760, max=23560, per=1.99%, avg=19380.92, stdev=1943.75
    lat (usec) : 2=0.04%, 4=0.01%, 10=0.16%, 20=10.69%, 50=18.22%
    lat (usec) : 100=12.63%, 250=42.20%, 500=9.99%, 750=1.46%, 1000=0.75%
    lat (msec) : 2=3.29%, 4=0.53%, 10=0.04%
  cpu          : usr=1.00%, sys=3.05%, ctx=1279344, majf=0, minf=1416
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=1280000/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: io=5000.0MB, aggrb=971905KB/s, minb=971905KB/s, maxb=971905KB/s, mint=5268msec, maxt=5268msec

Disk stats (read/write):
    dm-0: ios=0/1247022, merge=0/0, ticks=0/217499, in_queue=230737, util=100.00%, aggrios=0/1280000, aggrmerge=0/0, aggrticks=0/222904, aggrin_queue=231809, aggrutil=99.06%
  nvme0n1: ios=0/1280000, merge=0/0, ticks=0/222904, in_queue=231809, util=99.06%