ATA over Ethernet

From Wikipedia, the free encyclopedia

ATA over Ethernet (AoE) is a network protocol developed by the Brantley Coile Company, designed for accessing ATA storage devices over Ethernet networks. It gives the possibility to build SANs with low-cost, standard technologies.

AoE does not rely on network layers above Ethernet, such as IP, UDP, TCP, etc. This means that AoE is not routable over LANs and is intended for SANs only. An alternative to iSCSI, the AoE specification is 8 pages compared with iSCSI's 257 pages.

1 Operating system support
2 Hardware support
3 Related Concepts
4 See also
5 External links

[edit] Operating system support

The following OS provides native ATA over Ethernet support:

OS	Date	Release
Linux kernel [1]	2005-03-01	2.6.11

Coraid [2] provides device drivers for FreeBSD [3].

AoE support is currently being beta tested for the Solaris Operating Environment, Sparc and x86.

A European based storage company, Rocket Division Software has added AoE support into their StarPort Storage Controller for Windows, which is an iSCSI and AoE Initiator, RAM disk and Virtual DVD drive.

[edit] Hardware support

The Coraid [4] company offers an ATA over Ethernet hard disk called EtherDrive.

Also the vblade program makes it possible to export hard disks using inexpensive computers running Linux. Two independent implementations of vblade exist: A userspace one (part of aoetools package) and another one implemented as a linux kernel module.

[edit] Related Concepts

Although AoE is a simple network protocol, it opens up a complex realm of storage possibilities. To understand and evaluate these storage scenarios, it helps to be familiar with a few concepts.

[edit] Block Storage

The ATA in "ATA over Ethernet" is a kind of wire protocol for disk drives. Data is written to the disk and read back from it in one or more fixed-size chunks of data called blocks.

The ATA over Ethernet protocol simply puts ATA commands into low-level network packets, so that an Ethernet network effectively replaces the cable that goes to the disk drive. Just as blocks of data can go to a disk drive through a ribbon cable in a computer, they can also go over ethernet cables.

The ribbon cable doesn't care what is inside of the blocks of data, and neither does AoE. You can read and write any blocks of data you want, but most of the time, a filesystem is used to organize the data.

[edit] Filesystems on Block Storage

Traditionally, a filesystem has been used on a computer that is the sole user of its hard disk. Filesystems like ext3, XFS, and NTFS were designed with this assumption in mind.

When you use AoE, using an Ethernet network in the place of a cable, it's possible to violate that assumption. For traditional filesystems, that is a dangerous thing to do, leading to filesystem corruption or kernel panics.

Cluster filesystems avoid the assumption that only one computer has access to the block device. They are designed to allow multiple members of a cluster of computers to use a shared block device safely, by coordinating their actions.

Examples of cluster filesystems are GFS and OCFS2.

[edit] Disk Drives

The AoE target is a hard disk, or something that appears to be a hard disk. A few key points regarding hard disks are ...

Access speed is dependent upon disk RPM (rotational latency), head movement speed (seek time), magnetic density, accuracy of head alignment on the track, data location on disk, and interface electronics
Random disk access speed is entirely dominated by seek time
- Completely random access is 1000 times slower than sequential access
- Single disk sequential access is typically 50-80 MB/s
- RAID generally improves sequential and random access performance
- Host OS and disk filesystem try to store data sequentially to help the disk perform.

[edit] Ethernet

The wikipedia article on Ethernet networks has more information, but for AoE it is helpful to keep in mind the points listed below.

AoE packets use MAC addresses to indicate the source and destination. MAC addresses are the Ethernet-level addresses that are required for ethernet to work. A MAC address only works inside a single ethernet broadcast domain.
Current Ethernet hardware features include flow control, which helps to minimize the need for packet retransmission.
Ethernet uses Cyclic redundancy checks to ensure that packets arrive intact.

[edit] Network Storage

When using an Ethernet network to access block storage, there are several potential advantages:

It is easier to add storage capacity.
The amount of storage is practically unlimited.
Access can be controlled by creating ad hoc ethernet networks.
Common commodity hardware can be used.
Data backup may be easier.
Data may be shared instead of being confined to one computer.

[edit] Config String

The AoE protocol provides a mechanism for host-based cooperative locking. When more than one AoE initiator is using an AoE target, they must cooperate. The hosts need a way to avoid interfering with one another as they use and modify the data on the shared AoE device.

The hosts can simply communicate, using, for example, TCP/IP to send messages to one another. Through communication, the hosts can agree which host has the right to access particular blocks on the shared storage.

Another option is to use the storage device itself as the mechanism for determining the access of particular hosts. The AoE protocol includes a "config string" feature. If more than one host tries to set the config string, only one actually succeeds. The other host receives an error. At that point, the "winner" is decided and the winner can proceed to establish an access policy.