What is NVMe?

NVMe is short for Non-Volatile Memory Express, a protocol that makes solid-state drives (SSDs) run faster, and is gaining popularity among enterprise users.

To simplify our understanding of what NVMe is, let’s make an analogy—assuming you just bought a sports car that can reach 400 kilometers per hour, three to four times that of your old car. The only problem is that ordinary roads are not allowed to run at such speeds, and urban roads are usually limited to tens of kilometers per hour, often taking into account the flow of traffic. If you want to run faster, you need to change your way.

This scenario is a bit like the situation in the storage industry after manufacturers introduced flash-based SSDs. Flash is many times faster than traditional mechanical hard drives (depending on the device you use), but earlier devices used the same SATA or SAS as hard drives to connect to the system and networks. While these interfaces are sufficient for the performance of HDDs, they pose a bottleneck for SSDs, especially for enterprise big data analytics and other data-intensive workloads.

These performance issues are driving people to look for better ways to connect SSDs to servers, thus creating NVMe.

1. What is NVMe?

The NVMe protocol standard is overseen by NVM Express, a consortium of more than 100 organizations dedicated to developing faster protocols to improve the performance of non-volatile storage. The organization is led by a board of 13 companies including Cavium, Cisco, Dell EMC, Facebook, Intel, Micron, Microsemi, Microsoft, NetApp, Samsung, Seagate, Toshiba Memory and Western Digital.

HDD vs. SATA vs. NVMe in Maximum Theoretical Speeds

    NVMe is officially defined as "an open set of standards and information to fully unleash the advantages that non-volatile storage can provide in all types of computing environments from mobile terminals to data centers. NVMe is designed from the ground up to provide high bandwidth and low latency storage access for current and future NVM technologies."

    NVMe is an interface specification for connecting storage to a server via the PCI Express bus. Simply put, it allows SSDs to communicate faster with host systems and helps alleviate bottlenecks when flash is connected to the system via SAS or SATA originally designed for HDD.

    2. NVMe is much faster than SATA or SAS

    The NVMe controller improves performance in several different ways. One is the use of the PCIe bus, which directly connects the storage to the system CPU, eliminating some of the necessary steps of SATA and improving overall performance.

    In addition, NVMe SSDs achieve parallelism to a large extent, greatly improving throughput. When data is transferred from storage to the server host, it goes into a row or queue. But traditional SATA connection can only support one queue and receive 32 data at a time. Going back to the analogy, it's like a single lane that can accommodate 32 cars.

    NVMe storage supports up to 64,000 queues, each with 64,000 entries. In other words, it's like going from a single-lane road to a 64,000-lane road, each lane accommodating 64,000 cars.

    Just as our analogy says that urban roads generally have speed limits of tens of kilometers per hour, SATA and SAS connections also have speed limits. For SATA, the maximum theoretical transmission speed is 6.0 Gbps (In practice, the maximum transmission speed is much lower), which is equivalent to setting an upper limit for the speed of the SATA SSD. Beyond a certain limit, even using faster flash has no effect on the overall performance of the system, as SATA connections can cause bottlenecks.

    SATA SSD provides enough performance to perform routine tasks, and given its low price, most consumers prefer SATA SSDs to NVMe SSDs.

    However, companies typically process far more data than consumers so that the latency of SATA connections can be a problem.

The Intel 750 Series is one of the SSD supporting NVMe.

    3. Only SSD runs on NVMe

    Since NVMe is much faster than SATA, it seems like a good idea to use NVMe with HDD to improve performance. But keep in mind that NVMe stands for non-volatile memory interface specification, which is specifically designed for non-volatile storage such as NAND flash (although it can also be used for newer non-volatile memory such as 3D XPoint).

    When the system reads data from the HDD, it can only read one piece of data at a time, because it must rotate to the correct physical location of the first data block, then rotate again to the correct position of the second data block, and so on. However, flash and other non-volatile storage technologies don’t have mobile components, which means that the system can read data from many different locations simultaneously. This is why SSDs can take advantage of the parallelism provided by NVMe, whereas HDDs cannot.

    4. PCIe is related to NVMe, but they are not the same thing

    For many people, the most confusing part of NVMe is its relationship with PCIe. Some vendors use NVMe as tags to refer to their SSDs, others use PCIe tags, what’s more, some vendors seem to use these terms interchangeably.

    Although PCIe and NVMe are closely related, the two terms refer to slightly different technologies. PCIe can be regarded as the physical part of the system. When you plug an NVMe SSD into the server, you need to connect it through a PCIe slot.

    In contrast, NVMe is a protocol that is a set of hardware and software standards that allow SSD to use the PCIe bus. It can be said that NVMe is the language that allows the storage device to connect to the server, but PCIe is the actual physical connection.

    5. NVMe-oF connects SSD to the network

    This article focuses on the standard NVMe that directly connects the SSD to the server, while the NVM Express organization also released the NVMe over Fabric (NVMe-oF) specification that connects the non-volatile storage for block storage to the network. According to the organization, NVMe-oF defines a general architecture that supports a series of storage network structures for NVMe block storage protocols over network structures. This includes enabling front-end interfaces in storage systems, extending to a large number of NVMe devices and extending the accessibility distance between NVMe devices and NVMe subsystems within the data center.

    The NVMe-oF specification is about 90% the same as the NVMe specification; however, it does use a different transport mapping mechanism. NVME-oF also has two different variants: one for RDMA and the other for fibre Channel.

    The NVMe-oF specification is also much newer than the NVMe specification, and although some vendors have announced support for this technology, few actually sell NVMe products. It is expected that this technology will become more common in the coming years.