The Virtual Storage Guy

Is NetApp Missing from your Datacenter?

On behalf of NetApp, I’m writing to you with a simple proposition:

For your next storage purchase, you should consider NetApp.

Things move at a fast, if not exponential pace in the data center. The advent of virtual infrastructures and cloud computing has introduced a new set of requirements that redefined the value proposition of storage arrays within a cloud infrastructure stack.

The storage array that you ultimately select is responsible for maintaining most (if not all) of your organization’s intellectual property. This means that the success of your organization is dependent on the infrastructure that supports it and is tied to you ability to affect change in the quality of the storage you introduce.

Gone are the days when a shared storage platform simply provided high performance and data availability for a select set of business critical applications. Today, arrays MUST serve all of the data in the data center with greater than five (5) 9’s of availability while bringing to bear key enabling technologies such as: storage efficiency, data mobility, disaster resilience, continuous data protection, security and many, many more. Oh, did I mention that all this is predicated on a solution that must be repeatable, operationally consistent, non-disruptive, easy to manage, and able to grow with you and your business without adding unnecessary burdens to the adoption process?

What I have just articulated are array-specific requirements I have seen across the board – whether you are a small to medium business, a medium to lower-end enterprise, or a full on enterprise class customer. In every customer engagement that I have, these are always topics of conversation, and to you my fellow reader, I can only hope that many of these requirements are ringing true in your mind.

Now, if you do have a traditional storage array, then your existing vendor is likely a large and successful company with a long history of delivering big-iron solutions. Ideally, they have been there for you through the time of mainframes and open system through today’s emerging virtualization technologies, but you should ask yourself: “are they still offering a multitude of platforms under one banner, each offering a unique a specially tailored feature-set to address some portion of you virtual datacenter needs and ultimately your organization’s requirements?”

If you answered “no”, then let me thank you for being a NetApp customer ☺, otherwise, I encourage you to read on.

If you are open to new possibilities to extend your datacenter capabilities, especially for virtualization initiatives, or haven’t taken a closer look at NetApp’s technologies recently, then I’d like to offer you some compelling reasons as to why you should.

EMC Admits NetApp is the Leader

Chris Mellor of The Register recently published the article EMC races to catch up with NetApp, in which he covers a recent EMC webcast where EMC’s President and COO, Pat Gelsinger admitted EMC is hoping to catch up to NetApp in delivering storage efficiencies for virtual server and desktop environments.

I agree with Chris when he summarizes the current challenges facing EMC as...

“If EMC has misread the situation and NetApp's lead lies in its current feature superiority, such as block-level primary data deduplication, as well as in the general smoothness and integrated management and feature sets across the FAS array range, then NetApp will continue to enjoy its long run of above-average results; leaving EMC scratching its head and wondering just what the hell it has to do to rein in the Sunnyvale upstarts.”

UPDATE 2/11/11: I obtained the transcript and below is likely some of the core content that Chris is referring to in his article...

"Obviously you can look here and you can see there is one feature that NetApp, in this case, has that we don’t have. Block or primary block; a DDUP. And in that one feature today, we don’t support it in our product line. We will deliver that in the second half of this year, so we’ll close that last piece of competitive gap that we have. But collectively, it’s just another compression or another data-reduction technology" - Pat Gelsinger, VP & COO EMC

(If you are interested obtaining a copy of the transcript from the webcast go to www.callstreet.com or call 1-877-FACTSET)

Mr. Geslinger states NetApp has a technological lead, which EMC will try to close, yet I'm not sure he gets it. When discussing NetApp, Mr. Gelsinger states that the only advantage NetAp offers is the ability to run production workloads with the storage savings of data deduplication. The reality is NetApp is delivering much more than storage savings to virtual infrastructures.

I found the comments Mr. Gelsinger shared following his 'EMC will close the gap' statement rather interesting as he suggested compression and deduplication were merely alternative means to achieve storage efficiencies on par with that of deduce. This statement is only about 50% accurate. While both technologies provide capacity savings, dedupe is more ideal for all production use cases. It appears Mr. Geslinger is either misinformed as to the similarities and differences in these capabilities, or is attempting to obscure said differences in order to not raise questions around the 'EMC 20% Storage Savings Guarantee.'

I truly believe Mr. Gelsinger simply misspoke, and never intended to close with a message that was in stark contrast to his previous statement regarding the direction of EMC's engineering.

Mixing Messages Around Efficiencies

A Picture is Worth a Thousand Words

One portion of my role at NetApp is to share with the community the capabilities and value of our storage technologies, especially the ones engineered and optimized for cloud deployments. Doing so inevitably leads to skepticism from our competitors which makes sense as we are competing for the mind and market share of our customers.

Every storage platform has a set of similarities; I mean let's be frank, in the end they all serve data. Yet each also has a set of unique capabilities which may strengthen or weaken the value of a particular platform within a specific solution or architecture.

It must be tough as a potential customer, storage analyst, or technical enthusiast to be able to decipher the truth in claims such as, "NetApp is redefining the storage industry with storage efficiencies specifically designed for production workloads within cloud deployments."

It's with this question in mind that I would like to share with you what I expect is my last word on the subject in 2010...

(click on the image to see a larger view)

A special thanks to Doug for sending this link my way. Happy holidays!

Introducing Data Compression in ONTAP 8.0.1

It’s no secret; NetApp wants to sell you less storage than any other storage vendor and by forecasts from industry experts like IDC it is time we all begin enabling storage savings technologies throughout our datacenters. When NetApp was founded we unknowingly set out on this mission with the release of our first feature, the snapshot backup. It was a means for customers to take full backup copies of their data, yet each backup only consumed the amount of data that was unique to the data at the point in time the backup was created. The storage savings came from the sharing of blocks that were common between the production and backup data sets. Over the years we have continued to release new and enhanced storage savings capabilities for use in production, backup, DR, archival, and test & development data sets.

Suffice to say, storage efficiency is a driving theme in many of our technologies. With the release of Data ONTAP 8.0.1 we are extending our capabilities to include inline data compression and I’d like to spend a few minutes introducing you to our new offering. While it would be true to question the term industry leading as NetApp is one of the laggards in terms of adding support for compression, it would be rather myopic to discount our history of leading the industry with the vast number of storage savings technologies engineered for use with production workloads.

Once one considers the contents of this list it’s easy to understand why we lead the industry in reducing storage footprints in data centers through out the world:

Snapshot backups – logical full backups that only consume 4KB blocks that are unique between each backup

SnapMirror – Dedupe aware data replication that can be sent in a compressed form synchronously, a-synchronously, or in a semi-synchronous mode over fibre or IP.

RAID-DP – Provides data protection greater than RAID-10 with an overhead less equal to or less than RAID-5.

Thin provisioning – provisioning a logical unit of storage (FlexVol, LUN, or file) without pre-allocating the required storage.

Space reclamation – Ability to actually delete data that is marked as available for overwrite but still resides in the NTFS file systems of LUNs and VMDKs and on storage array. BTW – thin provisioning isn’t very compelling without space reclamation. It keeps thin, thin.

FlexClone – zero cost provisioning of FlexVols, Files, and LUNs (and sub-LUNs)

Data Deduplication – sharing physical 4KB blocks between dissimilar storage objects

Transparent Storage Cache Sharing – Dedupe aware cache that allows a 4KB block to be accessed by multiple external references (such as multiple VMs, DBs, etc.)

Platforms Supporting & Details of Data Compression

Let’s start with the basics; data compression is available in 8.0.1 with supported for the following arrays:

IDC says your data will grow and grow and grow...

Did you get a chance to read “The Digital Universe Decade - Are You Ready?” published by IDC (May 2010)? This report is a follow-up from the 2007 report titled, “The Expanding Digital Universe”. I ask this question as I'm surprised by the number of individuals who are unaware of its existence. In fact I don’t believe I met more than a dozen individuals at VMworld (US & Europe) or Insight (US & Europe) who had read either of these reports.

I should admit, that this report kind of flew under my radar as well. I owe thanks to The Skeptic’s Guide to the Universe podcast (which is one of my favorites and a recommendation to you) for covering this report in episode 251.

In short the Digital Universe is defined as all of the data that exists today and these reports attempt to measure the capacity of data in existence today while also providing a forecast for future growth. I'd suggest that these reports are fantastic reads and wanted to share with you a bit in order to prepare you for the amount of data storage capacity your infrastructure will be required to address.

Buckle up, it's gonna get wild!

Zettabyte Me!

The 2007 report estimated that the digital universe, all of the known data in the world, totaled 161 exabytes (or 161,000 petabytes). In addition, this report forecasted a growth of the digital universe to 988 exabytes by 2010.

FACT: VMware vSphere on NetApp is Faster and Greener

 

I'd like to ask you, do you think the following statements are generally considered to be true?

1. Virtualization is software, and hardware is simply hardware
2. RAID10 out performs all other RAID types that provide data protection (redundancy)
3. Fibre Channel provides greater I/O performance than NFS for virtualized workloads
4. Deduplicated data sets perform slower than traditional data sets

I believe that for most these statements are considered true and why shouldn’t they? These views originate from historical validation in our physical infrastructures. Moreover, most of us are not storage experts, intimately familiar with nuances in arrays architectures and capabilities. For most SAN & NAS storage is just a black box and thus storage is simply hardware; or as my good friend Duncan Epping likes to say, ‘just spinning tin’

 

Spinning Tin

Duncan’s comment of spinning tin led me to ponder, does hardware matter with server virtualization? On one hand no, it does not as the hardware layer has been abstracted to a point where customers at any time can non-disruptively migrate to a net-new platform.

Yet on the other hand hardware does truly matter.

The level of intelligence and capability in the hardware provides greater performance, solution sets, functionality, and points of integration (both direct like VAAI and indirect like dedupe).

Myth Busting: Storage Guarantees - Part II

I hope many of you read my recent post: Myth Busting: Storage Guarantees wherein I raised questions around the storage savings guarantee from EMC. The basic question is how can they deliver greater storage savings with fewer technical capabilities and use cases than what is available from NetApp. I believe this is a fair question and was hoping for some information sharing.

Now I don't expect most to be storage experts and to help raise the level of understanding I created the following chart.

Simply put, when one understands the technologies, their capabilities, and the parameters where each technology can be enabled; such a claim doesn’t appear to hold water.

As I have in the past, I extended an invitation to EMC to review the chart and to educate the public as to how the storage savings in their guarantee can be accomplished.

Unfortunately the only response (or compliment) the post received was from Chuck Hollis who shared a less than complimentary comment of "chartsmithing" in reference to the accuracy of the contents in the chart.

Honestly, I'm a bit concerned that the technical folks at EMC may have been asked to not engage in technical discussions. It appears as they want to communicate a message ofadvancedstorage technologies are the fundamentally the same from vendor to vendor, yet when asked to elaborate they becomedeafly silent. In my opinion, this behavior is like saying another hypervisor on the market today has the same capabilities as VMware ESXi, and one shouldn’t be concerned with the differences.

As for the lack of dialog, the silence speaks volumes…

The Lone Standout

To my surprise I recently learned of and read the post “Lies, Damn Lies, and Marketing…” from Richard Anderson, an engineer at EMC.

I commend Richard for his efforts and sharing his insight, and his level of professional courtesy. The post begins by verifying the content of the chart...

Myth Busting: Storage Guarantees

Way back at VMworld 2008 NetApp announced our 50% storage savings guarantee for virtual data centers. This guarantee basically stated that running VMware on NetApp would reduce one’s storage requirements by half and if we did not, the remaining storage was on us.

At the time these storage savings were powered by our block-level data deduplication and thin provisioning of LUNs and volumes and was specifically targeted for VMware and later Hyper-V deployments.

As we approach the two year anniversary of the guarantee I'd like to share that we have added a number of new technical innovations around storage savings technologies such as deduplicated and compressed replication, dedupe-aware cache, storage reclamation of deleted data trapped within guest file systems (i.e. NTFS formatted LUNs and VMDKs), etc.

I can also share that we have tens of thousands of customers running their clouds on NetApp and the average installation achieves 50%-70% storage savings with virtual servers and 90%+ with virtual desktops.

Imitation is the sincerest form of flattery

We were very flattered in 2009 when VMware established a 50% guarantee. While not a storage guarantee per se; I’m pretty sure that VMware's program was a smashing success (and probably didn't require any guarantee).

Data Deduplication and Data Compression in Data Ontap

Last week I published the post ‘Data Compression, Deduplication, & Single Instance Storage’ as a means to raise awareness around these different types of storage savings technologies and to provide some guidance as where each is best suited to be deployed.

I’d like to thank all that chimed in to share their thoughts and knowledge in the comments section of this post. I believe we can chalk up the discussion as a win for the readers.

(click image to view at full size)

It appears the last post led a few to revisit data deduplication with their NetApp arrays as I recieved a number of email and direct messages where the sender was (pleasantly) surprised to discover that Data Ontap 7.3.3 includes data compression to the list of the integrated storage savings technologies. (Storage features available can be view via the license command when connected to the FAS console).

Data Compression, Deduplication, & Single Instance Storage

Today I wrapped up several weeks of travel which included the Charlotte VMUG conference, NetApp's Foresight engineering event, and a number of customer and technical partner meetings. During these travels a small number of individuals would use the term data deduplication as any technology which reduces the amount of storage capacity required to store a data object (file, LUN, VM, etc...)

When this situation arose I would asked these individuals to share their understanding of data deduplicaiton. It became clear that they did not have a solid understanding of the differences between data compression, deduplication, and single instance storage, nor the benefits and considerations of each technology.

I guess one could summarize by saying these technologies are simply different means to achieve the same goal, the reduction of the storage footprint. However, this oversimplified answer does not provide any information which assists a data center architect in determining where each technology delivers the best benefit. This last point, is the goal of this post.

As we proceed I will refer to my trusty ruler as a means to visibly 'measure' storage savings obtained with each of these technologies.

Transparent Storage Cache Sharing– Part 2: More Use Cases

In part one of this series I introduced you to Transparent Storage Cache Sharing and introduced the use case of serving virtual machines and their binaries. The post received some great feedback and allowed me to realize that I wasn’t as clear as I had hoped to be in beginning this conversation.

Before we continue the conversation by looking at additional use cases, I think I should clarify a few details around traditional storage caches in order to effectively communicate the advantages of TSCS enabled storage arrays.

It’s fair to say that storage caches are available in different sizes, provide various predictive cache algorithms or means of read ahead in order to increase total storage performance. In their operation the cache stores data for use by a single VM in hopes that any subsequent requests from this same VM can be satisfied from cache before accessing disk. Any subsequent requests from any other VM cannot utilize the data cached for the first VM.

To put it another way, traditional array caching mechanisms lack transparency. They lack the ability to share a single reference between discrete objects such as two VMs.

TSCS provides such transparency and as a result discreet client requests can be served from a single instance of data in the array’s cache. This ability to serve multiple requests from a single copy of data provides a significant increase in array performance by serving more data requests from cache. This benefit provides a significant decrease in disk IO requests.

This capability is also available with any storage protocol be it SAN (FCoE, FC, iSCSI) or NAS (NFS, CIFS).

For today’s post I’ve added an I/O cache ratios to my images. This ratio depicts the number of discreet items that are served by a single cached item. I am hoping this addition will allow me to better depict the I/O gains provided by TSCS.

I’d like to point out that Linked Clones deliver a similar type of transparency via software. For the sake of this post I’d like to table the technical comparison of IO sharing via software and storage arrays for a later post. Fair enough?

Transparent Storage Cache Sharing– Part 1: An Introduction

I expect most of my readers are familiar with Transparent Page Sharing (TPS). For those in need of a refresh, TPS increases the over all performance capabilities of an ESX/ESXi cluster by ensuring efficient use of server memory. The TPS process achieves this efficiency by eliminating redundant virtual machine memory pages by remapping any duplicate reference back to a single page. The real ‘magic’ in TPS is its ability to transparently share the same pages (aka data) between multiple VMs.

Transparent Page Sharing is a key enabler in allowing the hypervisor to scale memory resources beyond the physical constraints.

This week I’d like to introduce you to Transparent Storage Cache Sharing (TSCS) and how it significantly changes the storage design and architecture in virtual infrastructures by allowing a single block of data to be accessed transparently by multiple virtual machines.

In this series, I will introduce you to the details of TSCS and review the benefits it delivers in a number of real-world use case scenarios.

SPEC SFS2008 Verifies You Can Run Faster on Fewer Disks with PAM

NetApp is the only storage vendor focused on reducing storage hardware footprints. Evidently discussions surrounding the use of storage saving technologies in a production environment must be upsetting to the traditional storage array manufactures as anytime I discuss the technologies that enable such environments the comments sections of my posts inevitably end up reading (and smelling) like the zoo after the monkey cage has endured a pooh-flinging outburst.

It is my premise that the attempts of the traditional legacy storage array vendors to dissuade customers from considering and implementing storage savings technologies is a self-serving and not a community driven interest. Maybe my statement is a bit inflammatory but just consider the following statements regarding the use of data deduplication from a well-known traditional storage manufacture VP...

”Typically, when storage admins run into I/O density problems, they have two fundamental approaches: more disks, or faster disks.”

“By spreading the load on more spindles, I/O density can be reduced. However, you now require more storage, power, cooling, etc. -- which kind of tends to defeat the whole premise of data deduplication to begin with.”

“The other approach is to use faster disks -- either disks that spin faster, or perhaps enterprise flash drives. Invariably, these cost more as well -- again tending to defeat the whole premise of attempting to save money through data deduplication.”

Statements like this are clearly meant to caution customers against the use of ‘risky’ new concepts (reduction of storage footprints) and the primary enabling technology (data deduplication). So is the information shared in this warning accurate?

Run Everything Virtualized and Deduplicated : aka Chuck Anti-FUD

Last night I got caught in what I would classify as a tweet-fight with several of my chums over at EMC on the role storage arrays play within a virtual datacenter. The conversation was pretty intense and at one point I posted the following string of tweets…

@sakacc @chuckhollis – Guys EMC & NetApp just look at virtualization differently. Allow me to explain…

@sakacc @chuckhollis - VMware allows customers to share CPU, memory, and network ports among multiple VMs allowing a reduction in servers

@sakacc @chuckhollis - Cisco allows customers to share ports among multiple connection protocols thus reducing network and storage switches

@sakacc @chuckhollis - NetApp allows customers to share disk capacity at a sub VM level among multiple VMs which reduces total storage

@sakacc @chuckhollis - EMC provides shared disk access to multiple VMs. Shared access is not shared resource usage; this is not virtualized

@sakacc @chuckhollis – VMware, Cisco, & NetApp want customers to purchase less hardware. What is EMC’s plan to reduce their HW footprint?

As you can see my premise is clear, for customers to be successful in their virtualization efforts, they must virtualize the entire datacenter this includes servers, networks, and storage.

Who decided that server and network hardware should be reduced while the shared storage footprint is permitted to grow uncontrollably?

The NetApp $1 Million Dollar Virtualization Challenge

Today we announced the $1 million dollar virtualization challenge contest. In this program we are looking for a non-NetApp customer who is willing to migrate their existing VMware infrastructure to a $1 million NetApp solution, if they are willing to publicly share the results of their migration. The winning submission will make out pretty well as they will receive $850K in storage, $100K in support, and $50K in software at no cost.

VCE-101: What's Next with Deduplication?

Welcome to the latest post in our series entitled ‘Virtualization Changes Everything’

In our last post we discussed the CapEx savings provided by data deduplication of both production storage capacity and storage array cache. Deduplication is fundamentally changing the data center and it appears that there’s more on the horizon. I’d like to share with you what I call ‘Data Center Ethernet Acceleration’.

Overview

“"Some call it a software mainframe others call it cloud, it depends on when you were born" – Steve Herrod, CTO, VMware

When discussing multiple applications on a single hardware platform I prefer the term ‘software mainframe’. I think this is due more to my time working with VMware products rather than my age. I still recall my introduction to this concept back in 2001.

VCE-101: Deduplication: Storage Capacity and Array Cache

Welcome to the first session in a series of blog posts entitled ‘Storage 101 - Virtualization Changes Everything’

Data deduplication was originally introduced in Data ONTAP 7.2 in July of 2006 as a technology that enhanced the NetApp Disk-to-Disk (D2D) backup offerings. Coincidentally VMware released VI3 around the same time in June of 2006, and shortly thereafter customers began using these two technologies together to virtualize their servers and storage footprint.

For today’s post we’re going to dive into the realm of storage efficiency and discuss data deduplication and it’s impact in production storage capacity and storage array cache specifically within a virtual data center.

The Format of this Series

As I stated when I decided to kick off these posts I want to share these topics with a wide audience and in order to do so each session will begin with a high level overview of the technology and its value followed by the technical details.