Tag Archive for storage

I Like Big Files and I Cannot Lie

You other vendors, can’t deny,
When an array walks in with an itty bitty waste [-ed capacity],
And many spindles in your face
You get sprung, want to pull up tough,
‘Cause you notice that storage was stuffed!

Ok… I’ll stop now! I’m just a bit sad and always wanted an excuse to to use that as a post opener! 🙂

There is a certain, quite specific type of customer whose main requirements revolve around the storage of large data sets consisting of thousands to millions of huge files. Think media / TV / movie companies, video surveillance or even PACS imaging and genomic sequencing. Ultimately we’re talking petabyte-scale capacities – more than your average enterprise needs to worry about!

How you approach storage of this type of data is worlds apart from your average solution!

The Challenges of “Chunky” Data

Typical challenges involve having multiple silos of your data across multiple locations, with different performance and workload characteristics. Then you have different storage protocols for different applications or phases in their data processing and delivery. Each of those silos then requires different skills to manage, and different capacity management regimes.

Sir Mixalot likes big files

On top of that, for the same reason as we moved away from parity groups in arrays to wide striping, these silos then have IO and networking hotspots, wasted capacity (sometimes referred to as trapped white space) and wasted performance, which cannot be shared across multiple systems.

Finally (and arguably most importantly), how do you ensure the integrity, resilience, and durability of this data, as by its very nature, it typically requires long-term retention?

Ideal Solution

What you really need is a single storage system which can not only scale to multi-petabyte capacities with multiple protocols, but is reasonably easy to manage, even with a high admin to capacity ratio.

You then need to ensure that data can also be protected against accidental, or malicious file modification or deletion.

Finally, you need the system to be able to replicate additional copies to remote sites, as backing up petabytes of data is simply unrealistic! Similarly, you may want multiple replicas or additional pools outside of your central repository which all replicate back to the mothership, for example for ROBO or multi-site solutions where editing large files needs to be done locally.

As my good friend Josh De Jong said recently:

Of course, the biggest drawback of using this approach is that you have one giant failure domain. If something somehow manages to proverbially poison your “data lake”, that’s a hell of a lot of data to lose in one go!

DellEMC Isilon

During our recent Tech Field Day 12 session at DellEMC, I was really interested to see how the DellEMC Isilon scale-out NAS system was capable of meeting many of these requirements, especially as this is a product which can trace its heritage all the way back to 2001! In fact, their average customer on Isilon is around 1PB in size, and their largest customer is using 144PB! Scalability, check!

The Isilon team also confirmed that around 70% of their 8,000+ customers trust the solution sufficiently to not use any external backup solution, trusting in SnapshotIQ, SyncIQ and in some cases SmartLock, to protect their data. That’s a pretty significant number!

One thing I am not so keen on with the Isilon (and to be fair, many other “traditional” /  old guard storage vendor offerings) is the complexity and breadth of the licensing; almost all of the interesting features each have to have their own license. If the main benefit to the data lake is simplicity, then I would far rather have a single price with perhaps one or two uplift options for licenses, than an a la carte menu.

In addition, the limit of 50 security domains provides some flexibility for service providers, but then limits the size of your “data lake” to 50 customers. It would be great to see this limit increased in future.Data Lake

The Tekhead Take

Organisations looking to retain data in these quantities need to weigh up the relative risks of using a single system for all storage, versus the costs of and complexity of multiple silos. Ultimately it is down to each individual organisation to work out what closest matches their requirements, but for the convenience of a single large repository of all of your data, the DellEMC Islion still remains a really interesting proposition.

Further Info

You can catch the full Isilon session at the link below:
Dell EMC Presents at Tech Field Day 12

Further Reading

Some of the other TFD delegates had their own takes on the presentation we saw. Check them out here:

Disclaimer: My flights, accommodation, meals, etc at Tech Field Day 12 were provided by Tech Field Day, but there was no expectation or request for me to write about any of the vendors products or services.

If you use Flash Storage, do NOT miss this Storage Unpacked Podcast!

Silicon Wafer

Just a very quick post today…

I don’t often write blog posts either responding to, or recommending things, as I usually just fire them out on Twitter or LinkedIn.

That said, I enjoyed this particular podcast episode so much, I felt I needed to share it with my [two] subscribers via a post!

 

Storage Unpacked

I have been listing with great interest to Chris Evans and Martin Glassborow’s new Storage Unpacked podcast for the past couple of months. If you are either a storage afficionado, or even a generalist, i highly recommend it.

The particular episode I am referring to is Talking NAND Flash with Jim Handy, specifically Part One, released 6th Jan this year.

As someone who follows the storage industry with interest, this episode was a great insight as to the history and decisions which have led us to where we are with flash storage today, as well as some fascinating facts and figures.

I don’t want to preempt or spoil the episode, but for example did you know:

  • Nanometer scale productions actually mean working at the scale of millionths of a millimeter
  • Fabrication plants cost around $8 Billion each to make, and the machines involved in creating the chips cost about $100 million each!
  • It takes 3 months to create a single chip from the raw materials!

These, and other interesting things can be found on the episode, below – I highly recommend you check it out (and dont forget to subscribe to their podcast)!

Talking NAND Flash with Jim Handy – Part One

Further Info

I listen to about 20+ podcasts on a regular basis (the one and only good thing about commuting every day to the office!). I need to do an updated article on them, but in the mean time, here is a list of some of my recommendations:

The Tekhead Top 10 Tech Podcast Perfect Playlist Picks

Now that’s what I call… Tech Predictions 2017

predictions

At this time of year, it is customary to look back at the past 12 months and make some random or not-so-random guesses as to what will happen over the coming 12. As such, what could be more fitting for my final post of 2016?!

Here’s a few of my personal best, worst, and easy guess candidates for 2017…

Tekhead Predictable Tech Predictions 2017

Easy Guesses

Come on Alex, even Penfold could have predicted these!

  • AWS will continue to dominate the cloud market, though the rate at which they deploy new features will start to slow (over 1000 a year is pretty unsustainable!). Their revenues will continue to grow at gangbuster rates, however their market share will be slightly eroded as people experiment more with their competitors too.
  • Microsoft Azure will grow massively (not quite 100% but not far off it). Their main growth will probably be in hosting enterprises and typical line of business applications as people move their legacy junk into the cloud. The recent announcements of the Single Instance VM SLA of 99.9% will definitely accelerate this as customers will feel less include to refactor their applications for cloud.
  • Distributed everything!
  • Docker will start to become more mainstream production and less Dev/Test.
  • Google will kill off at least one popular service with multiple millions of users.
  • The homelab market will reduce as people do more and more of their studying in the cloud.
  • Podcasting will become the new blogging (if it hasn’t already!)
  • DellEMC will continue to hack off bits of its anatomy to pay back that cheeky little $67Bn debt.
  • I continue to use memes as a crutch to make my otherwise lifeless articles marginally more interesting!obvious
Best Guesses

Its on the cards… maybe?

  • Google will continue to be ignored by most enterprises for Cloud IaaS. They will gain some reasonable growth in the web application space after another mass marketing activity to developers, ISVs and hosters.
  • Oracle grows Cloud revenues 50% or more but market share remains small. Their growth is mainly driven by IaaS revenue as customers begin to move their workloads to be closer to their data in the Oracle PaaS and SaaS services.
  • There will be no major storage company IPO in 2017, i.e. over $200m.
  • Many storage startups will run out of funding and die on the vine (depressing I know!). Their IP will be snapped up by the old guard storage companies in the proceeding fire sales…
    fire-sale
  • 3D XPoint will begin to creep into storage arrays by the end of the year, fuelling another storage VC funding bubble for at least another 12 months for any company who claims to have an innovative way to use it.
  • A major cloud provider suffers a global outage.
Worst Guesses

These probably won’t happen, but if any of them do, I’ll claim smugly that I knew they were always going to!

  • Pure Storage will make an acquisition of a storage startup to create their third product line, perhaps a secondary storage company (i.e. not just all flash) along the lines of Cohesity.
  • Cisco will buy a storage company. They will be more successful at integrating it than they were with Whiptail! (Which wouldn’t be difficult… 😮 )
  • Spanning a single application over multiple clouds becomes a real possibility, as one or more startups come out of stealth to provide innovative ways to span clouds. Nobody buys into it, except maybe for DR.
  • Tekhead.it becomes the most read blog in the world in 2017
  • Cats take over the planet and dogs are forced to form a rebel alliance which is ultimately victorious when a chihuahua takes out the entire cat leadership in one go, with a stolen reaper drone.Cats vs Dogs
  • Jonah Hill wins Strictly Come Dancing, narrowly defeating Frankie Boyle and Charlie Brooker in the final.
And finally…

Here’s wishing you all an awesome, fun and prosperous 2017!

Scale-Out. Distributed. Whatever the Name, it’s the Future of Computing

Scale Out

We are currently living in the fastest period of innovation in the technology space which there has probably ever been. New companies spring up every week with new ideas, some good, some bad, some just plain awesome and unexpected!

One of the most common trends I have seen in this however was described in a book I read recently, “The Second Machine Age” by Erik Brynjolfsson & Andrew McAfee. This trend is that the majority of new ideas are (more often than not) unique recombinations of old ones.

Take for example the iPhone. It was not the first smart phone. It was not the first mobile phone, the first touch screen, or the first device to run installable apps. However, Apple recombined an existing set of technologies into a very compelling product.

We also reached a point a while back where clock speeds of CPUs are no longer increasing, and even CPUs are scaling horizontally. Workloads are therefore typically being designed to scale horizontally instead of vertically, taking advantage of the increased compute resources available whilst avoid being locked to vertically scaling clock speeds.

Finally, another trend we have seen in the industry of late is inexpensive and low power CPUs from ARM, being used in all sorts of weird and wonderful places; often providing solutions to problems we didn’t even know we had. Up until now, their place has generally been confined outside of the data centre. I am, however, aware of a number of companies now working on bringing them to the enterprise in a big way!

So, in this context of recombination, imagine then if you could provide a scale-out storage architecture where every single spindle had its own compute directly attached. Then combine many of these “nano-servers” together in a scale-out JBOD form factor on subscription pricing, all managed from a Meraki-style cloud portal… well that’s exactly what Igneous Systems have designed!

Igneous Systems Nano-Servers

One of the coolest things about scaling out like this, is that instead of a small number of large fault domains based around controllers, you actually end up with many tiny fault domains instead. The loss of any one controller or drive is basically negligible within the system and replacements can be sorted at the convenience of the administrators, rather than panicking about replacement of components asap. Igneous claim that you can also scale fairly linearly, avoiding the traditional bottlenecks of a dual controller (or similar) system. It will be interesting to see some performance benchmarks as they become available!

It’s still early days, so they are doing code deployments at some pretty high rates, around every 2 weeks, and to be honest I think there is a bit of work to be done around clarity of their SLAs, but in general it looks like a very interesting platform, particularly when pricing is claimed to be as low as half the price of Amazon S3.

Now as you might expect from a massively distributed solution, the entry point is not small, typically procured in 212TiB chunks, so don’t expect to use it for your SMB home drives! If however you have petabyte-scale data volumes and are looking for an on-prem(ises!) S3 compatible datastore, then its certainly worth looking at Igneous.

The future in the scale-out space is certainly bright, now if only I could get people to refactor their single-threaded applications!… 🙂

Further Info

You can catch the full Igneous session at the link below – it certainly was unexpected and interesting, for sure!

Igneous Systems Presents at Tech Field Day 12

Further Reading

Some of the other TFD delegates had their own takes on the presentation we saw. Check them out here:

Disclaimer: My flights, accommodation, meals, etc at Tech Field Day 12 were provided by Tech Field Day, but there was no expectation or request for me to write about any of the vendors products or services.

%d bloggers like this: