newbie question: difference between fileio and blockio in IET

[Yongjian Zhang]

Hi all,

I have a newbie question about the difference between fileio and
blockio when it comes to IET.

To my own understanding, in a general context, fileio supports the IO
on a file basis, thus block operation is hidden from the user of
fileio. This IO style is like the NFS. And blockio means all the IO is
based on block number, and this is sort of under the file system
level.

If that is correct... then I'm confused about the difference between
them in the IET context and what I found by google seems to be
inconsistent...

I tried setting targets with fileio. And before I make the disk
partition as a target, I have built an ext3 fs on it. But when the
partition is exported to an iSCSI initiator (open-iscsi) , the
initiator does not recognize the fs and fdisk -l shows that the disk
does not have a partition table and etc. So it seems that iSCSI
initiator treat the target as a blockio device.. then how does fileio
actually work in iSCSI?

I saw a question on the IET Wiki "When should I use "fileio" vs
"blockio" ?" but I couldn't find the link to the answer... Search for
"blockio" on the IET Wiki yielded "no results"...

And according to the results found by google, some say that fileio in
IET is like NFS. But in my own experiment it seemed not to work in
that way.

Some say that iSCSI does not support a fileio style, but only blockio.
And the difference is that whether the target host fs cache is
involved, that is fileio stye uses the target host fs cache while
blockio does not. Further, some of these folks believe that blockio
has a higher performance.

Others say the IET does not support blockio anymore... Thus fileio is
the only way to go...

So I'm a little confused here...

Could anyone maybe tell the whether fileio and blockio are both
supported by IET? What are the theories behind them and how are they
supposed to be used to have the best performance?

Sorry for so many questions..

Any help would be highly appreciated!

Jack

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
Iscsitarget-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel

[Ross S. W. Walker]

Yongjian Zhang [mailto:[hidden email]] wrote:

>
> Hi all,
>
> I have a newbie question about the difference between fileio and
> blockio when it comes to IET.
>
> To my own understanding, in a general context, fileio supports the IO
> on a file basis, thus block operation is hidden from the user of
> fileio. This IO style is like the NFS. And blockio means all the IO is
> based on block number, and this is sort of under the file system
> level.

Not really.

Fileio uses the VFS layer to access it's backing store so it can
utilize regular files on file systems as well as block devices and
IO is cached in the page cache.

Blockio goes direct to the device, bypassing the VFS layer, so it
can only work with block devices directly.

Fileio provides good performance on systems with lots of RAM
but because it is memory mapped all IO is broken into 4k chunks
when going to disk, the system does a good job of combining them
for asynchronous IO, but not synchronous IO. This can lead to
performance issues in situations where either IO sizes are large
enough and fast enough to overwhelm the system in IOPS or the IOs
are >4k and synchronous.

Blockio talks directly to the storage drivers bypassing page cache
which means that the only caching taking place is what happens on
the storage controllers and drives. This will allow IOs of
arbitrary size to be passed to the storage controllers in a zero
copy method. Think of it as directly tieing your network interface
to your disk controllers. While no IO is cached it provides good
performance for large IO sizes and excellent latency for random
and synchronous IO.

Couple of examples:

Database storage: blockio
Since databases typically provide their own memory cache of data
and when they do go to disk it is usually for random writes to data
files.

Virtual server storage: fileio
Virtual machines typically load the OS in memory using asynchronous
sequential IO, then sit there issuing mostly random reads to the
same locations again and again.

File server storage: fileio or blockio
Files typically are read in whole via asynchronous sequential reads
and then are rewritten in whole with asynchronous sequential writes.
As a file system becomes more fragmented this may become more random
reads, but writes still tend to be sequential. File systems though
provide their own cache, local page cache, so double caching might
not make sense.

> If that is correct... then I'm confused about the difference between
> them in the IET context and what I found by google seems to be
> inconsistent...
>
> I tried setting targets with fileio. And before I make the disk
> partition as a target, I have built an ext3 fs on it. But when the
> partition is exported to an iSCSI initiator (open-iscsi) , the
> initiator does not recognize the fs and fdisk -l shows that the disk
> does not have a partition table and etc. So it seems that iSCSI
> initiator treat the target as a blockio device.. then how does fileio
> actually work in iSCSI?

You set a partition or volume as an iSCSI target, but understand
that the initiator looks at it as a whole disk drive which will
need a MBR/GPT on it to work.

Try creating the MBR/GPT and partitions/file systems from the
initiator side. Don't try to mount it on the target until it
is logged off from the initiator or it will corrupt. But if
you do need to mount it on the target use kpartx to make
sub-partitions out of the partition/volume and mount those.

> I saw a question on the IET Wiki "When should I use "fileio" vs
> "blockio" ?" but I couldn't find the link to the answer... Search for
> "blockio" on the IET Wiki yielded "no results"...

The wiki needs a little TLC, it's been neglected for too long.

> And according to the results found by google, some say that fileio in
> IET is like NFS. But in my own experiment it seemed not to work in
> that way.

Don't confuse iSCSI which presents block devices with NFS/CIFS
which presents file systems. The two are completely different
technologies.

You cannot share an iSCSI target between multiple initiators
without utilizing a third party clustering product.

> Some say that iSCSI does not support a fileio style, but only blockio.
> And the difference is that whether the target host fs cache is
> involved, that is fileio stye uses the target host fs cache while
> blockio does not. Further, some of these folks believe that blockio
> has a higher performance.

Fileio uses the file system cache or page cache as it is called.

Sometimes fileio is faster than blockio (async IOs or 16K IOs
or less) and sometimes blockio is faster than fileio (sync IOs or
32K IOs or more). It really depends on the workload, and that's
why we have both types.

> Others say the IET does not support blockio anymore... Thus fileio is
> the only way to go...

Hmmm, no, we still fully support blockio and continue to work on
and improve both.

> So I'm a little confused here...
>
> Could anyone maybe tell the whether fileio and blockio are both
> supported by IET? What are the theories behind them and how are they
> supposed to be used to have the best performance?

Both are fully supported and I hope I helped clear your questions
on when to use one or the other.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.


------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
Iscsitarget-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel