Archive for Tech Field Day

Swordfish – A Standard by Any Other Name Would Smell As Sweet

SNIA_SwordfishLogo

Whether it’s the IEEE, the ISO or any other, we live in a world governed by standards. This has the positive impact in allowing interoperability of devices and elements, but at the same time has the unfortunate side effect of hampering the development of new technologies which conflict with those standards, even if their adoption would ultimately provide a better outcome for everyone!

At the same time, many organisations (read: vendors) opt out of these standards and introduce their own. This is great for the vendor as it is tailored to their requirements and products, but it doesn’t help the customer or their lowly sysadmin who has to then implement a load of additional tooling to manage these products. Take S3 as an example; AWS took one look at what was out there in the market, decided that none of the standards met their requirements, so wrote their own!

The key seems to me to be finding a balance, where you implement a standard, but make it extensible, such that individual vendors can add additional data or functionality over and above the baseline. This means that you can always support the “lowest common denominator” for everyone.

So what is Swordfish?

Funnily enough, the folk from SNIA (The Storage Networking Industry Association) have implemented precisely this with one of their latest standards releases, Swordfish. Specifically, this defines the standards for APIs used to manage storage devices in a consistent fashion, regardless of vendor or indeed storage class (for example software based hyper-converged solutions are supported by it, as well as block, file, object, etc!).standardsThey have achieved this by taking the existing SMI-S standards and refactoring them into a simplified model which is client, not vendor oriented, and based on a REST API model, JSON (the current industry favourite for almost all data interchange) and OData. Not only that, but they have achieved this and agreed the standards with their many members in less than 12 months. By comparison to your average RFC from the IEEE that’s lightning fast! 😮

Now this is not to say that your typical vendor is going to throw out everything they have today, but if they begin to run these APIs in parallel, I could see this eventually becoming the defacto standard for all storage management. In addition, SNIA have confirmed that if 2-3 or more vendors have a requirement for the same additional fields (which they will initially have to implement via extensions), then SNIA will ratify them within weeks. Truly an agile methodology for standards!

The Tekhead Take

This seems to me to be a pragmatic approach to a difficult problem. Keeping vendors happy, whilst trying to make life easier for storage consumers and administrators by bringing storage management into the twenty-first century!

Despite being a relatively dry subject matter, I was actually quite interested and impressed with this innovation! People will still need dedicated local storage for many years to come, and these standards will help to enable them to manage storage in a more consistent fashion. Who knows, it may even promote more competition!

Want to Know More?

I was fortunate enough to meet the team from SNIA last year at their Colorado HQ, with Storage Field Day 13. One of the speakers (industry veteran Rob Peglar) also recently appeared as a guest on the Storage Unpacked podcast – an episode well worth a listen too!

Anyway, you can catch the session here:

SNIA Presents at Storage Field Day 13

SNIA have published a load of information on the standards here:

http://snia.org/swordfish

Finally, some of the other SFD13 delegates had their own thoughts on the session and standards as a whole. You can find them here:

Disclaimer/Disclosure: My flights, accommodation, meals, etc, at Storage Field Day 13 were provided by Tech Field Day, but there was no expectation or request for me to write about any of the vendors products or services and I was not compensated in any way for my time at the event.

Docker – State of the Nation (aka Observations of a Brit)

Docker Logo

It may surprise you to learn that Docker is actually quite old now (at least in Startup terms!), having released the first version of their very cool software in March 2013!

Throughout that time, Docker (the company) have moved at a fairly rapid pace in terms of feature and etween

ug releases, with an average of a point release about every quarter and minor releases every month (or more)!

Whilst sitting here awaiting my flight to VMworld Europe 2017, where there are MANY MANY MANY (MANY!) sessions on Docker, Photon, Kubernetes, etc on the session schedule, I am prompted to consider Docker’s rise to popularity, and finish off a post I begun a few months back after Tech Field Day 12!

Well come on Galbraith… get on with it then!

My experience in UK IT industry over the last (nearly) 15 years has taught me a few things, one of which is that whenever new technologies begin serious adoption in the US, it usually becomes popular in the UK within 2-3 years. That said, this number has been squeezed down a little in the past few years as companies move towards more agile development and deployment methods. Fail fast is becoming the mantra of many more organisations, though some people I speak to still wake up with night sweats at even the thought!

The first time a customer asked me about Docker in the UK was over 3 years ago, yet in all that time, people I talk to outside of the social media bubble many of us live in have been virtually silent about it; that is until now. Docker is becoming a weekly conversation topic now with a lot of organisations I talk to, with a many people wanting to jump on board the band wagon. The switch from an operating system-centric view of the world, to a more application and service-oriented (or should that be microservice-oriented) view of the world is becoming far more prevalent in my experience.Docker Swarm

Drivers to Docker Adoption

So what is it about this Docker stuff which seems to be catching the attention of people I talk to? A few common themes I hear are:

Automation of code deployment pipelines (CI/CD) to increase business agility
I think this is probably the number one driver to Docker adoption for people I talk to. Automation of CI/CD pipelines has become so common now, it is almost becoming the norm. Yes, it is tricky to do this with more traditional applications, but it certainly isn’t impossible. Using containers as the delivery mechanism for your application provides very consistent and repeatable outcomes. I mean, you can even get Oracle DB in a container now?!?!

That said, once you dockerise your applications there are many further challenges you will run into, including something as simple as how to apply your current security tooling, policies and proceedures to these new environments.

Maturity of the platform
The Docker code base and third party ecosystem has finally reached a point of maturity where many of the networking and storage issues of the past are beginning to reduce to within acceptable risk boundaries.

Improved cross-industry support
Following this maturity model, a swathe of vendors have put their names behind the Docker ecosystem; from VMware to Openstack, AWS to Azure, Google to Cloud Foundary, everyone is getting on board! You no longer have to buy support direct from Docker (the company), but can instead get it from your cloud vendor, along with a managed orchestration tier too, such as Docker Swarm, Kubernetes or Mesos!

Because Cloud
Yes, you can Dockers your existing applications for use on premises, but many organisations I speak to are using Docker as a method to allow their developers to write code on premises, test in their dev environments on prom or in the cloud, then deploy in a consistent fashion to their brand spanking new Production cloud platforms. PaaS solutions such as Azure WebApps and AWS Elastic Beanstalk are becoming a good option for customers who just want to write code, but for those who want that little bit more control, Docker gives them flexibility and consistency.to the cloud

CIO/CTO CV Padding
I hate to play the cynic, but I think there is definitely a significant percentage of CIOs/CTOs who are doing “digital transformations through containerisation and cloud” specifically to pad out their CVs and help them get a better gig.

This is otherwise known as a “Resume-driven IT Strategy”!

I am aware of one CIO who deliberately went to a cloud platform, even though it was significantly more expensive than a traditional managed hosting solution of a similar spec, when their business case and steady workload drew few, if any discernible benefits from the use of cloud.
CIO CV Padding When I hear people refer to technologies such as VMware vSphere as “Legacy” it really drives home to me the shift we are all going through, yet again, in the industry. This is another reason though which CIOs/CTOs/Heads of IT tell me they want cloud and containers. That said, I still struggle to find a single person who doesn’t have at least one physical server in their infrastructure, so just like the mainframe before it, I don’t think the hypervisor is going away any time yet!

The Tekhead Take

As expected the lag of a couple of years from the US to the UK in adoption of containers was apparent, but now is most definitely the time! Despite both positive and negative reasons for integrating it, Docker has become the part of the information technology zeitgeist in the UK…

Want to Know More?

I was fortunate enough to meet with the product team from Docker at Tech Field Day 12 towards the end of last year. It was a really interesting session which covered many of the enterprise networking and security features recently introduced to the platform, along with Docker’s new support offerings. I highly recommend checking it out!

Docker Presents at Tech Field Day 12

Some of the other TFD12 delegates had their own thoughts on the session and Docker as a whole. You can find them here:

Disclaimer/Disclosure: My flights, accommodation, meals, etc, at Tech Field Day 12 were provided by Tech Field Day / Gestalt IT, but there was no expectation or request for me to write about any of the vendors products or services and I was not compensated in any way for my time at the event.

Does Cloud Provide Infinite Storage Capacity and Retention?

cloud

I wrote last week about the challenges of long-term retention of data, and some of the architectural considerations and decisions we take in designing long-term backup or archive solutions. The follow-up question therefore is, does the cloud provide infinite storage capacity and retention?

“Cloud Integration”

One of the key themes which I have been seeing of late with many (if not all!) modern storage solutions, is some form of cloud integration. It seems to me that many vendors are trying to ensure they can tick the “cloud integration” check box in an RFP or RFI!

I recall one time at a previous organisation, our storage team did an RFP asking for an array which was capable of doing file presentation. The response in the RFP was “Yes”, but when this was dug into a bit further (after the fact), it turned out that this was only possible with an HA pair of custom vendor file gateways. In other words, not much better than building your own file server!

Anyway back to the point, this “RFP checkbox” mentality means that some vendors have a very tight cloud integration with multiple target replication options (such as DC to DC, DC to Cloud, Cloud to DC, Cloud to Cloud, etc), whilst others provide little more than lip service to cloud integration.

The best suggestion I can make in this scenario is to push your vendor for either a demo, a PoC, or a software copy of their array, if they have one. That way you can be absolutely sure that what is claimed, is indeed what you are looking for!

One Possible Solution… EMC Unity

One solution I believe falls more and more into the cloudy camp with each code release, is the new EMC Unity arrays, for which we were provided a briefing at the recent Storage Field Day 13 event.

What I found particularly interesting was that the arrays were natively capable of up to 256 redirect-on-write snapshots per volume, which sounds like a lot, but if you do one every 5 minutes then you will run out pretty fast! By utilising the EMC Cloud Tiering Appliance (a totally separate management interface today, which I really hope EMC fully integrate into the Unity pretty quick, as multiple panes of glass are no fun for anyone!), we can utilise any S3-compatible storage to provide UNLIMITED snapshots.unlimited snapshot retention

This is pretty cool if you have to provide very granular restoration points for your application data, as well as the ability to off-site at relatively low cost in a near-infinite data storage facility!

Sadly, you can’t currently run VMs directly from those snapshots in the cloud, but bearing in mind that EMC has a software only version of the Unity already available, I have a sneaking suspicion that there will be some engineering talent working on this as we speak! This would potentially provide the ability to snap and replicate your entire estate natively to S3 buckets in the cloud, then restore very quickly locally within that IaaS platform. Let’s hope I’m right!

Want to Know More?

EMC’s sessions on Unity, Scale-IO and Isilon were recorded and are now available to stream online:

Some of the other SFD13 delegates had their own thoughts on the session and EMC in general. You can find them here:

Disclaimer/Disclosure: My flights, accommodation, meals, etc, at Storage Field Day 13 were provided by Tech Field Day / Gestalt IT, but there was no expectation or request for me to write about any of the vendors products or services and I was not compensated in any way for my time at the event.

Long Term Data Retention – What do I do?

One of the more common requirements I come across on a day to day basis working with organisations across a broad spectrum of industries is the question of how to manage long-term data retention.

Frankly, I have massively oversimplified the question as there are many more nuances to it that this! Some of the questions, discussion points and potential solutions I see when trying to scope out and define a long-term data retention strategy are below. We assume in this case that we are talking about backing up application data, but the same can apply to file data, such as from a file server.

Long Term Data Retention – Questions, questions, questions?!

Like beautiful snowflakes, ultimately it always comes back to gathering the requirements for the individual business.

What are the regulatory and compliance requirements for long-term retention of data, and what are the consequences for loss of that data? In the new world, this could be pretty serious, especially with things like GDPR right around the corner. Escalating this up the business hierarchy can get buy in from other parts of the business to provide additional budget outside of IT, for a solution to meet the actual requirements, not just a botch job which will likely fail when put to the test.

How long is the actual data retention required? Looking at most current applications, if we are relying on being able to read back data in 7 years, current or future backup software may still work, but will we have the kit to read the tapes or data? If using spinning rust as a storage media, do we expect to be able to migrate data from one disk system to another easily in future, and if so, how does that impact things like encryption, capacity, deduplication and compression of that data?

What is it that we are trying to protect against? Deliberate or accidental deletion, total destruction of a server, array or DC, or perhaps we just need to be able to prove what your data looked like at a specific date / time.

How granular does the data need to be? For example do we need to be able to pull a file version from a specific week in the past X years? The more granular we need to get, potentially the more expensive. If we have controls in place to protect archive data against accidental / deliberate deletion, then we may not actually need to keep more than a few days or weeks of backups (as an example).

The use of FIM (File Integrity Management) tooling can be very helpful in this regard, especially for flat file structures. They can track all changes to your file system and if something is removed or updated, you could alert your server teams to investigate why and restore it from a recent backup.

Can the application or server prevent deliberate or accidental data deletion? If the application can be treated as, or write to, WORM storage (Write Once Read Many times), then the risk of data loss is further reduced, especially if that storage can be replicated off site. This doesn’t really help much with things like SQL databases, however!

Where is the archive data for the application or solution actually held? Is it within the live system (e.g. the live DB), or can it be exported onto a tertiary archive system where it becomes Read Only to all parties, including administrators? Even better, can the application export the data into a generic format, more likely to be readable in 25+ years time (such as CSV, text etc)? This provides quite a bit more flexibility in terms of future access and recovery options.

Does the application or server provide RBAC, and has it actually been implemented yet? If we minimise the number of people who could update or delete data (maliciously or accidentally), we minimise the risk of data loss.

What is the budget for the solution? All singing, all dancing, physical or software solutions can be great, but you may not be able to afford them.

Are we looking for an appliance-based solution which includes storage, replication, backup plugins, etc, or do you already have the HW and just need some software? This often, but not always, comes down to a time vs budget question. Do you want to spend your team’s time managing clunky backup software, or just buying an appliance which does half the work for you and is policy based?

What are your sovereignty requirements for the data, and would a cloud-based service be appropriate for your business? It can be very cheap to store data in something like S3 or blob storage, if the business accepts this and you don’t need to pull any of the data back very often (if at all).

How quickly is the data required when requested, how large is a typical access request, and how often are they needed? If this can be hours or days, then an offline or cloud solution may be appropriate, but anything where immediate access is required, is a different story.

Similarly, will we want to restore or access this data in the event of a DR, does this solution form part of our DR strategy? Perhaps it’s only required for access to much older data because you are replicating the most recent data to a DR facility!

As we can see, there are many, many, [many!] things to think about when considering long-term retention of data in a backup or archive solution.

What brought this up Alex?…

… I hear you ask!

I recently attended Storage Field Day 13, where we had a presentation from a backup vendor, StorageCraft, who has been in the SMB and mid-market space for many years, and it got me thinking!

The latest iteration of their backup software provides a local cache with cloud integration, and the added ability to spin up a DR environment in the event of an outage to your primary DC. A pretty nifty feature if you are legally able to store your data outside of your local environment (they currently have DCs in the US and EU only).

They can also create backups using their proprietary SPF file format, which has apparently not changed since its inception around 15 years ago. There is also no concept of a media server, as each server manages its own backups (albeit with the ability to use a central scheduler tool). This gets around the issue of backup compatibility, though may limit their ability to provide additional data services for the backup files, such as encryption, dedupe or compression, outside that of the storage targets they reside on.

This is what tickled my mental matrix into deploying my keyboard! 🙂

Want to Know More?

The session was recorded and is now available to stream online:

StorageCraft Presents at Storage Field Day 13

Some of the other SFD13 delegates had their own thoughts on the session and StorageCraft in general. You can find them here:

Dan Frith – StorageCraft Are In Your Data Centre And In The Cloud

Scott Lowe – Backup and Recovery in the Cloud: Simplification is Actually Really Hard

Disclaimer/Disclosure: My flights, accommodation, meals, etc, at Storage Field Day 13 were provided by Tech Field Day / Gestalt IT, but there was no expectation or request for me to write about any of the vendors products or services and I was not compensated in any way for my time at the event.

%d bloggers like this: