Tag Archive for Media

I Like Big Files and I Cannot Lie

You other vendors, can’t deny,
When an array walks in with an itty bitty waste [-ed capacity],
And many spindles in your face
You get sprung, want to pull up tough,
‘Cause you notice that storage was stuffed!

Ok… I’ll stop now! I’m just a bit sad and always wanted an excuse to to use that as a post opener! 🙂

There is a certain, quite specific type of customer whose main requirements revolve around the storage of large data sets consisting of thousands to millions of huge files. Think media / TV / movie companies, video surveillance or even PACS imaging and genomic sequencing. Ultimately we’re talking petabyte-scale capacities – more than your average enterprise needs to worry about!

How you approach storage of this type of data is worlds apart from your average solution!

The Challenges of “Chunky” Data

Typical challenges involve having multiple silos of your data across multiple locations, with different performance and workload characteristics. Then you have different storage protocols for different applications or phases in their data processing and delivery. Each of those silos then requires different skills to manage, and different capacity management regimes.

Sir Mixalot likes big files

On top of that, for the same reason as we moved away from parity groups in arrays to wide striping, these silos then have IO and networking hotspots, wasted capacity (sometimes referred to as trapped white space) and wasted performance, which cannot be shared across multiple systems.

Finally (and arguably most importantly), how do you ensure the integrity, resilience, and durability of this data, as by its very nature, it typically requires long-term retention?

Ideal Solution

What you really need is a single storage system which can not only scale to multi-petabyte capacities with multiple protocols, but is reasonably easy to manage, even with a high admin to capacity ratio.

You then need to ensure that data can also be protected against accidental, or malicious file modification or deletion.

Finally, you need the system to be able to replicate additional copies to remote sites, as backing up petabytes of data is simply unrealistic! Similarly, you may want multiple replicas or additional pools outside of your central repository which all replicate back to the mothership, for example for ROBO or multi-site solutions where editing large files needs to be done locally.

As my good friend Josh De Jong said recently:

Of course, the biggest drawback of using this approach is that you have one giant failure domain. If something somehow manages to proverbially poison your “data lake”, that’s a hell of a lot of data to lose in one go!

DellEMC Isilon

During our recent Tech Field Day 12 session at DellEMC, I was really interested to see how the DellEMC Isilon scale-out NAS system was capable of meeting many of these requirements, especially as this is a product which can trace its heritage all the way back to 2001! In fact, their average customer on Isilon is around 1PB in size, and their largest customer is using 144PB! Scalability, check!

The Isilon team also confirmed that around 70% of their 8,000+ customers trust the solution sufficiently to not use any external backup solution, trusting in SnapshotIQ, SyncIQ and in some cases SmartLock, to protect their data. That’s a pretty significant number!

One thing I am not so keen on with the Isilon (and to be fair, many other “traditional” /  old guard storage vendor offerings) is the complexity and breadth of the licensing; almost all of the interesting features each have to have their own license. If the main benefit to the data lake is simplicity, then I would far rather have a single price with perhaps one or two uplift options for licenses, than an a la carte menu.

In addition, the limit of 50 security domains provides some flexibility for service providers, but then limits the size of your “data lake” to 50 customers. It would be great to see this limit increased in future.Data Lake

The Tekhead Take

Organisations looking to retain data in these quantities need to weigh up the relative risks of using a single system for all storage, versus the costs of and complexity of multiple silos. Ultimately it is down to each individual organisation to work out what closest matches their requirements, but for the convenience of a single large repository of all of your data, the DellEMC Islion still remains a really interesting proposition.

Further Info

You can catch the full Isilon session at the link below:
Dell EMC Presents at Tech Field Day 12

Further Reading

Some of the other TFD delegates had their own takes on the presentation we saw. Check them out here:

Disclaimer: My flights, accommodation, meals, etc at Tech Field Day 12 were provided by Tech Field Day, but there was no expectation or request for me to write about any of the vendors products or services.

VMworld Europe 2015 Day Four and Closing Thoughts

VMworld

As with every year, the final day of VMworld was a pretty subdued atmosphere. In the main this is due to the number of people who head home early, combined with the those left looking distinctly hungover from the VMworld party on Wednesday night! Fortunately I remained reasonably coherent all night, other than in the act of losing my voice somewhat due to the volume, (Yes, I am turning into a grumpy old man who likes his virtual slippers by the fireside) and the inevitable VMworld lurgy, which in my case kicked in during rather than after the event!

The morning was largely spent shooting the breeze, chewing the fat, grinding our axes and many other classic metaphors, with the guys in the bloggers area. Needless to say we set the world to rights, defined the product strategy VMware should be taking for the next 20 years, and redefined the UK tax system so that it was fairer for all involved… ahem

I managed to squeeze in a couple more sessions over lunch time, including a great group discussion on NSX and vCD integration led by Ray Budavari and Wade Holmes. The vast majority of people in the room came from service providers, and of those only one was using NSX without vCD, so it appears that there is life in the old dog yet!

NSX & vCD

One of the interesting points from the session is that it looks as though the different editions of NSX will eventually be rationalised. vSphere will likely be the “favourite child” of NSX, getting new features first etc, but multi-hypervisor support will continue to be a feature in the future. Probably quite reassuring if you have already made a significant investment in the technology, though upgrades are likely to be a bit of a concern as they bring together the different Code Streams (groan).

After that I managed to catch up with the inimitable Alastair Cooke, key member of the #vBrownbag posse, who gave me some excellent advice for my upcoming trip to Storage Field Day 8. It’s always a pleasure to catch up with Alastair. He was a massive help in passing my VCAP-DCD4 back in the day, so if you don’t already subscribe to his excellent blog, I highly recommend you check it out!

After that it was time to hop on the shuttle bus and had for the airport for my flight home…

 

Closing Thoughts

Another VMworld Europe comes and goes, and much like the US edition there weren’t a huge number of life-changing announcements to write home about. Of the things which were in play however, Cloud Native Apps were most definitely front and centre! VIC is a great option for those organisations looking to get their feet wet in the container space, whilst being assured of the security that comes with being backed by the support resources of a company like VMware.

Final Thoughts

If you’re a software vendor or enterprise with some chunky legacy custom applications and you are considering going down the CNA route, just remember you don’t need to boil the ocean! Instead of spending the next 3 years cloudifying™ and microservicing™ your app for some major release, think about starting small.

  • Target new application functionality to be built with microservices at the core.
  • Find the performance bottlenecks in the existing application and rewrite that code to be able to scale out in a microservice architecture.
  • Think about how deep you want to go with your new microservice architecture? Design it at a business function/task level, which can be improved and iterated over time.
  • Consider for each microservice, what you would want to happen if that service fails. For example if your search services fail, customers can still access content. This is fine for most sites, but it wouldn’t work for Google, so make sure you drill down on your requirements! One example of this is that if the recommendation engine in Netflix fails for some reason, users will still get a default list of recommendations, rather than a big fat error message!
  • Don’t forget about security! Microservices are awesome, but they introduce a whole new level of complexity…
  • Above all, always bear in mind Baguley’s law (see below)!

For those customers wanting to scale beyond the 10k container limit imposed by vCenter itself, Photon will be an option too, though I have a sneaking suspicion that customers of that scale may look at doing something a little more open/custom anyway.

VMware Photon

When it comes to VMware’s other applications you can definitely see some decent forward momentum, particularly in products which have been bought in and integrated, such as vRA and NSX. In many cases I think the migration processes to the newer versions are all a bit too “rip and replace unless you have a 100% vanilla install”, but as the products mature further I think this will become less of an issue. It will definitely cause a few customers some pain in the short term though, especially if they just went out and spent thousands on PSO to implement the current version, only to have to redo half the work to upgrade! I guess you could continue to hold off until a later version if you want to reduce hassle, but if you never set a foot on the path you’ll never actually reach your destination!

It was great to catch up with many old and new faces at the event, especially at the vExpert event and in the bloggers area. It’s funny how you feel you kind of know people pretty well before you’ve even met them, if only via your 140 character interactions, so when you’re actually face to face for the first time it’s like catching up with old friends!

Until next year…

A Few Links

I was kindly invited to do a wee interview for VMworld TV by Eric Sloof at the vExpert party on Monday night. If you want to take a look, the link is below. I won’t embed it as the thumbnail image looks like I’m having some kind of embolism!
https://www.youtube.com/watch?v=N8CXTxvtb-I

If you didn’t manage to attend the event, it’s not too late to take advantage of some of the awesome content and sessions. VMware post a significant number of the most popular sessions on YouTube for free public consumption. Did I mention they’re free? Andreas Lesslhumer has kindly put together a summary list of all of the available videos from both US and EU events here:
http://www.running-system.com/vmworld-2015-general-sessions-and-technical-sessions-available-online/

Needless to say I’m very much looking forward to next year’s event already. If you want to attend, I suggest you preregister now to be notified when tickets become available:
https://www.vmworld.com/en/pre-register.html

 

Walk a mile in another man’s shoes…

I know it has now become something of a tradition to post how far you walked during the week, so here are my stats. I would caveat however that I am not a lazy bar steward, I’m 6’7″ tall, so I don’t need to take as many steps as other people! 🙂

Day Steps ~Distance
Monday 8564 6 km
Tuesday 7374 5km
Wednesday 10274 7km
Thursday 8356 6km

It now becomes obvious why everyone says bring comfortable shoes! 🙂

 

Quote of the Week

This undoubtedly goes to VMware CTO Joe Baguley, during the CNA Panel session on Day three:

%d bloggers like this: