Tag Archive for Cloud

Amazon AWS Tips and Gotchas – Part 6 – AWS Dedicated VPCs

Continuing in this series of blog posts taking a bit of a “warts and all” view of a few Amazon AWS features, below are a handful more tips and gotchas when designing and implementing solutions on Amazon AWS, including Dedicated VPCs.

For the first post in this series with a bit of background on where it all originated from, see here:
Amazon #AWS Tips and Gotchas – Part 1

For more posts in this series, see here:
Index of AWS Tips and Gotchas

AWS Tips and Gotchas – Part 6

12. AWS Dedicated VPCs

Just a quick one this week, specifically something to watch out for otherwise you risk running up a scary bill very quickly!

When you create a new VPC, you have the option to create it as Default or Dedicated as per the screenshot below:

AWS Dedicated VPCs

Now here’s the rub… if you select dedicated VPC, this will actually cause every single EC2 instance from then on to be created on dedicated hardware (what AWS call single-tenant hardware, i.e. dedicated physical servers!) by default, within that VPC.

Also note that as per the Dedicated Instances KB article, “You can’t change the instance tenancy of a VPC after you create it”.

In other words, if you find you have created your VPC as a dedicated one, you will have to destroy and re-create everything within that VPC to get it back to default (i.e. multi-tenant/shared compute).

AWS Dedicated VPCs invoiceAnyhoo, I said it was just a quick one this week…

Find more posts in this series here:
Index of AWS Tips and Gotchas

Amazon AWS Tips and Gotchas – Part 7 – AWS EMR, Spot Instances & PGs

Amazon AWS Tips and Gotchas – Part 5 – Managing Multiple VPCs

Continuing in this series of blog posts taking a bit of a “warts and all” view of a few Amazon AWS features, below are a handful more tips and gotchas when designing and implementing solutions on Amazon AWS, based around VPCs and VPC design.

For the first post in this series with a bit of background on where it all originated from, see here:
Amazon #AWS Tips and Gotchas – Part 1

For more posts in this series, see here:
Index of AWS Tips and Gotchas

AWS Tips and Gotchas – Part 5

11. Managing Multiple VPCs & Accounts

Following on from the previous post, let us assume that instead of just talking about public services endpoints (e.g. S3, Glacier, etc), and instead we are talking about environments with multiple VPCs, possibly multiple accounts, and the potential addition of Direct Connect on top.

AWS VPC VPCs

Why would you do this? Well, there are numerous reasons for logically separating things such as your dev/test and production environments from a security and compliance perspective. The one that people sometimes get hung up on is why would I want more than one account? As it goes, some AWS customers run many tens or even hundreds of accounts! Here are a few examples:

  • The simplest answer to this is so that you can avoid being “CodeSpaced” by keeping copies of your data / backups in a second account with separate credentials!
  • Separation of applications which have no direct interaction, or perhaps minimal dependencies, to improve security.
  • Running separate applications for different business units in their own accounts to make for easier LoB billing.
  • Allowing different development teams to securely work on their own applications without risking impact to any other applications or data.
  • With the mergers and acquisitions growth strategy which many companies adopt, it is fairly common these days for companies to be picked up and bring their AWS accounts and resources with them.
  • Lastly, a very common design pattern for compliance is to use a separate account to gather all of your CloudTrail and other audit logs in a single account, inaccessible to anyone except your security team, and therefore secure from tampering.

The great thing is that with consolidated billing, you can have as many accounts as you like whilst still receiving a single monthly bill for your organisation!

We will now look at a few examples of ways to hang together your VPCs and accounts, and in the majority of cases, you can effectively consider the two as interchangeable in so far as the scope of this post.

Scenario A – Lots of Random VPC Peering and a Services VPC

This option is ok for small solutions but definitely does NOT scale and is also against best practice recommendations from AWS. As mentioned in the previous section, transitive peering is also not possible unless you are somehow proxying the connections, so if you are looking to add Direct Connect to this configuration, this just simply isn’t going to fly.

Imagine that all of the blue dotted arrows in the following diagram were VPC peering connections! Aaaaargh!

AWS VPC VPCs

Option B – Bastion Server in Services VPC

If each of your VPCs is independent, and you only need to manage them remotely (i.e. you are not passing significant traffic between many different VPCs, or from AWS to your MPLS, then a services VPC with a bastion server may be a reasonable option (hub and spoke):

http://docs.aws.amazon.com/AmazonVPC/latest/PeeringGuide/peering-configurations-full-access.html

In this example, you could push a Direct Connect VIF into VPC A and via your bastion server, manage servers in each of your other VPCs. This would NOT be appropriate if your other servers / clients on premises wanted to access those resources directly, however, and is more likely in the scenario where each VPC hosts some form of production or dev/test platform which is internet facing, and this is effectively your management connection in the back door.

You might also potentially aggregate all of your security logs etc into the bastion VPC.

AWS VPC VPCs

Scenario C – Full Mesh

This is like a neater version of Scenario A. Holy moly! Can you imagine trying to manage, support or troubleshoot this?

AWS VPC VPCs

Even something as simple as managing your subnets and route tables would become a living, breathing nightmare! Then what happens every time you want to add another VPC? shudder

If you require this level of inter-VPC communication, then my first question would be why are you splitting the workloads across so many dependent VPCs, and where is the business benefit to doing so? Better to look at rationalising your architecture than try to maintain something like this.

Scenario D – Lollipop Routing

If you absolutely must allow every VPC to talk to most or even every other VPC, and the quantity of VPCs is significant then it may be worthwhile looking at something more scalable and easy to manage.

This one is more scalable from a management perspective, but if I am honest, I am not massively keen on it! It feels a bit like AWS absolving themselves of all responsibility when it comes to designing and supporting more complex network configurations. It could potentially also work out rather expensive as you could end up needing a fairly hefty amount of Direct Connect bandwidth to support the potential quantity of traffic at this scale, as well as adding a load of unnecessary latency.

I would prefer that AWS simply allowed some form of auto configured mesh with a simple tag/label assigned to each VPC to allow traffic to route automatically. If only such a technology existed or could be used as a design template!?! (sarcasm mode off – MPLS anyone?)

I am confident that at the rate AWS are developing new services, providing automation of VPC peering won’t be miles off (as suggested by the word “presently” in the following slide from an AWS presentation available on slideshare from last July (2015):

AWS VPC VPCs

In the meantime, we are left with something that looks a bit like this:

AWS VPC VPCs

When reaching this kind of scale, there are also a few limitations you want to be aware of:

AWS VPC VPCs

And Finally… NOTE: Direct Connect is per-Region

When you procure a direct connect, you are not procuring a connection to “AWS”, you are procuring a connection to a specific region. If you want to be connected to multiple AWS regions, you will need to procure connections to each region individually.

To an extent I can see that this makes some logical sense. Let’s say they allowed access through one region to others, if you have connections to a single region and that region has a major issue, you could end up losing access to all regions.

What would be good though would be the ability to connect to two regions, which would then provide you with region resilient access to the entire AWS network of regions. Whether this will become a reality is yet to be seen, but I have heard rumblings that there may be some movement on this in the future.

Wrapping Things Up

As you can see, getting your VPC peering and Direct Connect working appropriately, especially at scale, is a bit of a minefield.

I would suggest that if you are seriously looking at using Direct Connect, and need some guidance you could do worse than have a chat with your ISP, MSP or hosting provider of choice. They can help you to work out a solution which is best for your businesses requirements!

Find more posts in this series here:
Index of AWS Tips and Gotchas

Further Reading

Here are links to a few resources used in the writing of this post, worthwhile reading if you want to understand the subject more thoroughly:

Amazon AWS Tips and Gotchas – Part 6 – AWS Dedicated VPCs

StorageOS – An array based on containers? It’s like storage for millenials!

Last week I managed to catch up with the guys from StorageOS, a new container-based storage company, headquartered in London. I found out about them at a London Storage Beers event a few weeks ago, and my first question was, what the hell is container-based storage, and how does it work?!

They started from the premise (yes that’s actually the correct use of the word premise!), that if you want to build a storage system FOR containers, what better way to do it than to build it FROM containers. StorageOS therefore offer what they describe as “full enterprise storage array functionality, delivered by software, on a pay-as-you-go basis”. They also plan to offer a free-forever Developer tier, which includes everything except HA functionality which you would obviously need for production usage!

StorageOS Announcement

So the good news is, today (Monday 20th June 2016) StorageOS are announcing the release of their Beta at DockerCon, so you can now download and test out their new storage platform.

The StorageOS Stack

The StorageOS Stack

 

You can deploy this StorageOS software anywhere from bare metal to containers:

StorageOS - It's software, so it runs anywhere!

It’s software, so it runs anywhere!

Appliances for some of the larger clouds are in the works, but will not be available on day zero.

They can then consume any back-end storage, from SSD, HDDs and virtual drives, to EBS volumes, object stores, etc. You then pool all of capacity from all devices into a capacity pool, which is deduped, encrypted, and available across all nodes, and carve out volumes to present to systems like Docker through their own native Docker driver, or (slightly oddly) iSCSI / FC!!! They even have VAAI support in development!

Overall, I think it’s a pretty interesting product. At first look it feels a bit like a traditional array in a container package, much like if you containerised an enterprise app, then just utilised as a traditional array with some container plugins, instead of being very targeted and container-specific. StorageOS do have an OS driver to let you mount their volumes direct from containers, but there are other things out there today which do that anyway (e.g. Flocker).

I would say their messaging is a little inconsistent at the moment, and adding things like FC integration early on feels a bit odd if they’re positioning themselves as a container play. They do however state clearly that they’re targeting enterprises and want to make the on-boarding process as simple and friction-less as possible. I do worry that this “all things to all people” approach could be a wee bit risky at this early stage, and being more laser focused in the short to medium term would allow them to differentiate more.

StorageOS Cloud

The founders were very specific when they stated that they were building a clustered array with synchronous remote replicas, not a distributed storage array. Async replication is coming, which will be critical to maintaining performance in a hybrid cloud or multi-cloud setup. I really like the fact that you can stretch the same hybrid storage environment between your on-premises and cloud infrastructure using a single storage solution. This same solution can actually be used to span multiple public clouds as well, providing a resilient storage solution between say AWS and Azure, all of which is deduped and encrypted of course! This could be very interesting indeed, as customers look to protect their workloads from large public outages!

Finally, the StorageOS software is built (as you would expect these days) with APIs at the heart of everything. Even the modern GUI is really just based on API calls to the back end.

The Tekhead Take

Anyway, enough gabbing… It’s still early days, but the storage experience of the founders is certainly solid! Who better than ex-storage admins to provide a product that works well for storage admins?! I’d say there’s a good chance of this becoming a pretty cool product in the future, so definitely one to watch!

You can find a link to their website and beta sign up here:
http://storageos.com/index.php/product/

StorageOS hipster-approved storage

7 Reasons Why You Should Read The Phoenix Project

The Phoenix Project

I began reading The Phoenix Project with no preconceptions, other than having been told that it is a great book, and hearing it mentioned many times on Eric Wright‘s GC On Demand podcast.

Written by Gene Kim, Kevin Behr, and George Stafford, it is told as a first-person narrative from the perspective of Bill, a middleware team manager who is promoted into a senior IT management role for a business in jeopardy. Through his experiences and a guiding hand from another key character, together we work through the problems facing the business, the IT department and the individuals within.

The story is told in an easy to read, informal style, and I made quick work of it over the course of just a few days. I really enjoyed it on numerous levels:

  1. I recognised every single character in the book as somebody I have worked with (or indeed currently work with!). I guarantee you will feel the same!
  2. The book was pretty well written, and the story arc itself was compelling. I was really rooting for Bill to succeed in his endeavours! (But did he? You will have to read the book to find out!)
  3. The authors obviously have a great sense of humour! Quotes such as “Show me a dev who isn’t crashing production systems, and I’ll show you one who can’t fog a mirror. Or more likely, is on vacation.” had me laughing out loud on the train in front of other passengers!
  4. The book is approachable and not elitist. You could pick it up as a cable monkey or an IT director (or maybe even a Sales person!!!), and relate to the concepts and methods described.
  5. I learned a huge amount about different methods for handling and improving processes around WIP (Work in Progress), such as the Theory of Constraints or the use of Kanban boards (I am currently testing this with my pre-sales customer workloads using Trello, but I’m told Kanbanize is also very good). Resilience Engineering (think Netflix Simian Army) and numerous other techniques are also covered, along with the overarching “Three Ways” (very Zen!).
  6. I actually picked up a few key tips which could be applied directly to my pre-sales design and requirements gathering workshops with my customer stakeholders.
  7. Finally, it didn’t feel “preachy”, which is always a risk when trying to sell an idea / concept as your main theme and I was initially concerned that the book would be ramming DevOps culture down my neck throughout. This could not be farther from the truth, and the full DevOps concepts do not come into play until the story is almost complete. There are many lessons to be learned throughout the story, which could be applied to any organisation!

The Phoenix Project Cover

Here are another few choice quotes from The Phoenix Project, both humorous and insightful:

“The only thing more dangerous than a developer is a developer conspiring with Security. The two working together gives us means, motive, and opportunity.”

“How can we manage production if we don’t know what the demand, priorities, status of work in process, and resource availability are?”

“You just described ‘technical debt’ that is not being paid down. It comes from taking shortcuts, which may make sense in the short-term. But like financial debt, the compounding interest costs grow over time. If an organization doesn’t pay down its technical debt, every calorie in the organization can be spent just paying interest, in the form of unplanned work.”

“On the other hand, if a resource is ninety percent busy, the wait time is ‘ninety percent divided by ten percent’, or nine hours. In other words, our task would wait in queue nine times longer than if the resource were fifty percent idle.”

In case you hadn’t felt like I was positive enough about The Phoenix Project yet, I would say that this book should be provided as mandatory training to every person working in every IT department today, from the guys plugging in cables to the CIO!

If you do read and enjoy the book, I highly recommend also reading The Goal by Eliyahu M. Goldratt. I was a little surprised, to say the least, that this appears to be a very similar story, following a similar arc and some almost identical characters to The Phoenix Project. That said, I am half way through it at the moment and still thoroughly enjoying it, though I am not too worried about missing the movie version!

The Goal by Eli Goldratt CoverThe Goal delves even deeper into the Theory of Constraints and explains some of the tools we can use to mitigate, bypass or remove constraints in a system. All of these tools and methods can be applied as easily to IT as they can to production lines, which (without stating the bleeding obvious) is exactly the point of The Phoenix Project!

Anyway, if you want to do yourself a favour both in terms of your career development, but also a really compelling story and a thoroughly decent book, you could do a lot worse than spending £5 on the Kindle Edition of The Phoenix Project!

Where To Get Them

For anything technical, I like to buy ebooks these days for both portability and the fact that I wont be chopping down trees needlessly. Both of the above titles are available very inexpensively on Kindle:

And Finally…

Sincerest apologies for one of the most click bait-y blog titles I’ve ever posted! Even worse than this one. Honestly, I feel ashamed!

I’ll get my coat…

%d bloggers like this: