Tag Archive for storage

Without good Analytics you dont have a competitive storage product

Throughout my career, analysing storage utilisation for solution design and capacity management has not been an easy task! Even recently when I speak to customers about utilisation, they often don’t have the management tools in place on their legacy arrays or servers to be able to help us understand what their true workloads look like, or indeed often just basic statistics.

Gathering them is laborious at best, and almost impossible at worst. For example:

  • One previous major vendor I used to work with was only able to surface a small amount of basic throughput and latency data over the past 30 days or so, along with a bit of controller and port utilisation, through their Java-based BUI (Java version specific of course – I still shudder at the thought).
  • More recently another vendor I have used has a web based stats console which can aggregate multiple arrays, but they use a rather outdated method of visualisation which requires filling in a big form to get the stats generated and the produced graphs don’t include any kind of trending data or 95th percentile, etc.
  • Another vendor array I work with fairly regularly requires you to run an API call against the array which only provides you with the stats since the last time you ran it. By then running the API every 30 seconds to a minute, you can build up a body of stats over time. Not brilliant, and it’s a total pain to rationalise the exported data.
  • Even if you have the stats at the array, you need to then gather the same stats at the connected hosts, to ensure that they roughly correlate and that you don’t have any potential issues on the network (which is significantly more likely if say you are running storage and IP traffic on a converged network fabric).

In a word; clunky!

One of the things that struck me about many if not all of the vendors at Storage Field Day 8, was how much better the management consoles and analytics engines were than virtually all of those I have used in the past.

Several vendors use their dial home features to send the analytics back to HQ. This way the stats for individual customers as well as their customer base as a whole can be kept almost indefinitely and used to improve the product, as well as pre-emptively warning customers of potential issues through analysis of this “big data”. This also avoids customers having to spend yet more money on storing the data about their data storage!

Of those we spoke to, one vendor in particular really stood out for me; Nimble Storage. Their InfoSight platform gathers 30-70m data points per array, per day, which are uploaded to their central analytics platform and accessible via their very user friendly interface. It can produce a number of very useful graphs and statistics, send scheduled reports, and will even provide predictive upgrade modelling based on current trends.

Recently they have also added a new opt-in VMVision service which can actually plug into your vCenter server to track the IO stats for the VMs from a host / VM perspective as well, presenting these in conjunction with the array data. This will show you exactly where your potential bottlenecks are / are not, meaning that in a troubleshooting scenario you can avoid wasting precious time looking in the wrong place and all of the data is automatically rationalised into a single view, with no administrative effort required.

As certain storage array features are becoming relatively commoditised, it’s becoming harder for vendors to set themselves apart from the field. Having strong analytics and management tools is definitely one way to do this. So much so, I was compelled to tweet the following at the time:

Disclaimer/Disclosure: My flights, accommodation, meals, etc, at Storage Field Day 8 were provided by Tech Field Day, but there was no expectation or request for me to write about any of the vendors products or services and I was not compensated in any way for my time at the event.

Dell finds $67Bn down the back of the sofa – what to do with it? Buy EMC of course!

Dell Acquires EMC

This is just a quick post as I should be concentrating on VMworld right now, but (obviously) all everyone is talking about at the event is the momentous news that Dell are to acquire EMC for a record $67 Billion! There are so many possible implications from this event, so many questions which arise immediately and probably many more as the dust starts to settle. For example:

  • Dell have a very strong relationship with Microsoft, not only providing large quantities of infrastructure into MS corporate and Azure, but collaborating closely on products like the Microsoft Azure Pack etc. How do MS feel about a key partner buying one of their biggest competitors (VMware) and how will they react?
  • How many EMC and Dell products will be dead in the water within 6-12 months? If you are in the middle of your buying cycle right now, would you want to actually make a long term investment into either? This could be a great time for some of the other major players and startups to expand their market shares.
  • What is the future of VMware? They are still technically independent, though 80% owned by EMC. Would Dell want to maintain that position, buy up the remaining stock, or perhaps even sell off VMware to pay off a good chunk of the EMC purchase price? The market certainly didn’t seem too impressed as the VMware stock price dropped over 10% after the deal become public!
  • What will the likes of HP do now that their biggest competitor has just bought one of their biggest partners?
  • Will the SEC approve the merger (I am certainly no expert on US financial law, but I assume they will have some sort of say over this due to the scale of the deal)?
  • Maybe all of the above are moot, and everyone will continue to get along as Frenemies / Co-petitors (©2015 Jane Rimmer)… then again, maybe not?

People have been talking about Joe Tucci’s retirement for years now; I doubt anyone imagined it would end with this!

Looking Forward to Storage Field Day 8

Storage Field Day

I have been a fan of the Tech Field Day events for some time. They provide a really interesting approach to tech marketing and are a great way of keeping up with the latest developments in the industry, as tech startups and established players alike take to the stage to showcase / discuss (and often get grilled by the delegates about) their shiniest new toys and features.

One of the key services I see the guys at TFD providing (free!) to the community is helping to maintain our knowledge of the bewildering array (pardon the pun) of vendors and solutions that are available out there, in an easy to consume format. It’s important to keep up with current trends and releases in the storage arena, even if you only have vendor X or Y in your current environment. If only so that when your IT Director says to you he wants to go out and buy vendor Z you can have a sensible, fact based conversation as to why or why not to consider them (instead of the obvious knee jerk reaction which they will potentially be expecting!). In my case I’m just a massive geek who loves talking / learning / reading / writing about tech, so keeping up definitely isn’t a chore for me! 🙂

So with that in mind, I am very honoured and excited to have been invited to attend Storage Field Day 8 from the 21st to 23rd October this year. Thank you very much to Stephen Foskett (@SFoskett) and Claire Chaplais (@cchaplais) for the awesome opportunity!

I would consider myself an IT generalist with a penchant for virtualisation and storage. The thing that has really drawn my interest to the storage field has been the fact that it is one of the fastest moving parts of the industry today, with the most innovation and potential disruption from startups.

You don’t have to be an established player to be successful any more. The cost of entry when basing your solutions on Intel chips and white box chassis with a layer of cleverly written software is a heck of a lot cheaper than the custom hardware driven solutions of the past! As many companies have a wide selection of storage silos across their estates, it is also not so difficult to encourage them to try out your new solution to initially replace a single silo either. Lastly lets be honest, we all like an underdog, and there are quite a few underdogs nipping at the bellies of the 880lb storage gorillas as we speak!

Morpheus doesnt like high margin storage

For the past three years I have been working as a Solution Architect at Claranet, an independent pan-European managed services provider, designing hosting solutions for the mid-market; an interesting and challenging sector where aspirations sometimes exceed budgets. That said, I will try not to repeat the traditional service provider mantra of “Can I securely multi-tenant it?” and “Do you provide an Opex commercial model?” too much…

I am really looking forward to enabling my brain sponge and soaking up the vast combined knowledge of the delegates and presenters at the event (some of whom I listen to regularly on the highly recommended podcasts Greybeards on Storage and In Tech We Trust and all of whom are known for producing awesome community content), so be sure to check them out and follow them on twitter!

The list of vendors at SFD8 is extensive too… with some new names who only came out of stealth in the past year along with the more familiar ones, it should be a fascinating week!

SFD8 Vendors

You can join the live stream during the event, and recordings of all sessions are available after, all of which you can find here:
http://techfieldday.com/event/sfd8/

PS: Being half Saffa, half Scot I was a bit concerned I might miss some of the RWC 2015 action by being in the States during the semi final stage, but after spending this Saturday sitting in the stands during the (now infamous) SA vs Japan game, I’m sadly less concerned about that possible outcome now!

Tech Startup Spotlight – Hedvig

Hedvig

After posting this comment last week, I thought it might be worth following up with a quick post. I’ll be honest and say that until Friday I hadn’t actually heard of Hedvig, but I was invited along by the folks at Tech Field Day to attend a Webex with this up and coming distributed storage company, who have recently raised $18 million in their Series B funding round, having only come out of stealth in March 2015.

Hedvig are a “Software Defined Storage” company, but in their own words they are not YASS (Yet Another Storage Solution). Their new solution has been in development for a number of years by their founder and CEO Avinash Lakshman; the guy who invented Cassandra at Facebook as well as Amazon Dynamo, so a chap who knows about designing distributed systems! It’s based around a software only distributed storage architecture, which supports both hyper-converged and traditional infrastructure models.

It’s still pretty early days, but apparently has been tested to up to 1000 nodes in a single cluster, with about 20 Petabytes, so it would appear to definitely be reasonably scalable! 🙂 It’s also elastic, as it is designed to be able to shrink by evacuating nodes, as well as add more. When you get to those kind of scales, power can become a major part to your cost to serve, so it’s interesting to note that both x86 and ARM hardware are supported in the initial release, though none of their customers are actually using the latter as yet.

In terms of features and functionality, so far it appears to have all the usual gubbins such as thin provisioning, compression, global deduplication, multi-site replication with up to 6 copies, etc; all included within the standard price. There is no specific HCL from a hardware support perspective, which in some ways could be good as it’s flexible, but in others it risks being a thorn in their side for future support. They will provide recommendations during the sales cycle though (e.g. 20 cores / 64GB RAM, 2 SSDs for journalling and metadata per node), but ultimately it’s the customer’s choice on what they run. Multiple hypervisors are supported, though I saw no mention of VAAI support just yet.

The software supports auto-tiering via two methods, with hot blocks being moved on demand, and a 24/7 background housekeeping process which reshuffles storage at non-busy times. All of this is fully automated with no need for admin input (something which many admins will love, and others will probably freak out about!). This is driven by their philosophy or requiring as little human intervention as possible. A noteworthy goal in light of the modern IT trend of individuals often being responsible for concurrently managing significantly more infrastructure than our technical forefathers! (See Cats vs Chickens).

Where things start to get interesting though is when it comes to the file system itself. It seems that the software can present block, file and object storage, but the underlying file system is actually based on key-value pairs. (Looks like Jeff Layton wasn’t too far off with this article from 2014) They didn’t go into a great deal of detail on the subject, but their architecture overview says:

“The Hedvig Storage Service operates as an optimized key value store and is responsible for writing data directly to the storage media. It captures all random writes into the system, sequentially ordering them into a log structured format that flushes sequential writes to disk.”

Supported Access Protocols
Block – iSCSI and Cinder
File – NFS (SMB coming in future release)
Object – S3 or SWIFT APIs

Working for a service provider, my first thought is generally a version of “Can I multi-tenant it securely, whilst ensuring consistent performance for all tenants?”. Neither multi-tenancy of the file access protocols (e.g. attaching the array to multiple domains for different security domains per volume) nor storage performance QoS are currently possible as yet, however I understand that Hedvig are looking at these in their roadmap.

So, a few thoughts to close… Well they definitely seem to be a really interesting storage company, and I’m fascinated to find out more as to how their key-value filesystem works in detail.  I’d suggest they’re not quite there yet from a service provider perspective, but for private clouds in the the enterprise market, mixed hypervisor environments, and big data analytics, they definitely have something interesting to bring to the table. I’ll certainly be keeping my eye on them in the future.

For those wanting to find out a bit more, they have an architectural white paper and datasheet on their website.

%d bloggers like this: