Domain Migration to http://tekhead.it is Now Complete!

Tekhead Logo

This is just an uber quick reminder that as per my previous post, I have now updated the domain for the blog from http://www.tekhead.org to http://tekhead.it.

The changes went live tonight (27/02/2016) around midnight, and all previous blog paths are now 301 redirected to the equivalent address on the new site. Hopefully I won’t lose too much Google juice with the new address!

So, if you have any difficulties whatsoever accessing paths or content, please let me know via Twitter, and I would be very grateful!

Amazon AWS Tips and Gotchas – Part 1 – AWS Intro, EBS and EC2

Although I have been very much aware of AWS for many years and understood it at a high level, I have never had the time to get deep down and dirty with the AWS platform… that is until now!

I have spent the past three weeks immersing myself in AWS via the most excellent ACloud.Guru Solution Architect Associate training course, followed by a one week intensive AWS instructor-led class from QA on AWS SA Associate and Professional.

While the 100 hours or so I have spent labbing and interacting with AWS is certainly not 10,000, it has given me some valuable insights on both how absolutely AWSome (sorry – had to be done!) the platform is, as well as experiencing a few eye openers which I felt were worth sharing.

It would be very easy for me to extoll the virtues of AWS, but I don’t think there would be much benefit to that. Everyone knows it is a great platform (but maybe I’ll do it later anyway)! In the meantime, I thought it would be worthwhile taking a bit more of a “warts and all” view of a few features. Hopefully, this will avoid others stepping into the potential traps which have come up directly or indirectly through my recent training materials, as well as being a memory aid to myself!

pretty cloud AWS EC2 EBS

The key thing is with all of these “gotchas”, they are not irreparable, and can generally be worked around by tweaking your infrastructure design. In addition, with the rate that AWS develop and update features on their platforms, it is likely that many of them will improve over the coming months / years anyway.

The general feeling around many of these “features” is that AWS are indirectly and gently encouraging you to avoid building your solutions on EC2 and other IaaS services, Instead, pushing you more towards using their more managed services such as RDS, Lambda, Elastic Beanstalk etc.

This did originally start off as a single “Top 10” post but realised quickly that there are a lot more than 10 items and some of them are pretty deep dive! As such, I have split the content into easily consumable chunks, with a few lightweight ones to get us started… keep your eyes open for a few whoppers later in the series!

The full list of posts will be available here:
Index of AWS Tips and Gotchas

AWS Tips and Gotchas – Part 1
  1. Storage for any single instance may not exceed 20,000 IOPS and 320MB/sec per EBS volume. This is really only something which will impact very significant workloads. The current “recommended” workaround for this is to do some pretty scary things such as in-guest RAID / striping!

    Doing this with RAID0 means you then immediately risk loss of the entire datastore if a single EBS volume in the set goes offline for even a few seconds. Alternatively, you can buy twice as much storage and waste compute resources doing RAID calculations. In addition, you then have to do some really kludgy things to get consistent snapshots from your volume, such as taking your service offline. 
    In reality, only the most extreme workloads hit this kind of scale up. The real answer (which is probably better in the long term) is to refactor your application or database for scale-out, a far more cloudy design.
    amazon AWS EBS
  2. The internet gateway service does not provide a native method for capping of outbound bandwidth. It doesn’t take a genius to work out that when outbound bandwidth is chargeable, you could walk away with a pretty significant bandwidth bill should something decide to attack your platform with a high volume of traffic. One potential method to work around this would be to use NAT instances. You can then control the bandwidth using 3rd party software in the NAT instance OS.
  3. There is no SLA for EC2 instances unless you run them across multiple Availability Zones. Of course with typical RTTs of a few milliseconds at most, there is very little reason not to stretch your solutions across multiple AZs. The only time you might keep in one AZ is if you have highly latency sensitive applications, or potentially the type of app which requires a serialised string of DB queries to generate a response to the end user.

    In a way I actually quite like this SLA requirement as it pushes customers who might otherwise have accepted the risk of a single DC, into designing something more robust and accepting the (often minor) additional costs. With the use of Auto Scaling and Elastic Load Balancing there is often no reason you can’t have a very highly available application split across two or more AZs, whilst using roughly the same number of servers as a single site solution.

    For example the following solution would be resilient to a single AZ failure, whilst using no more infrastructure than a typical resilient on-premises single site solution:Teahead AWS Simple HA Web Configuration
    No DR replication required, no crazy metro clustering setup, nothing; just a cost effective, scalable, highly resilient and simple setup capable of withstanding the loss of an entire data centre (though not a region, obviously).

Find more posts in this series here:
Index of AWS Tips and Gotchas

Amazon AWS Tips and Gotchas – Part 2 – AWS EBS & RDS MS SQL

 

Docker Part 3 – HOWTO Create a Simple Python Web App in Docker

Docker Logo

If you’ve been following this series (last part here), we now have docker installed, but what do we do next? Create our first containers of course!

I think we need to make it a bit more interesting though as just creating containers is a bit meaningless, in real life we’re actually going to do something with them. The scenario is that we want a few copies of our simple python web application. To achieve this we need to use a few simple docker commands:

  • Create a new container
  • Install an application inside of it
  • Store it as an image
  • Duplicate it more than once and make these available to other clients
  • Test each instance to ensure they are unique and accessible

The good thing here is that all of the above steps are repeatable with whatever application you wish to install inside your containers. This is just a simple way to help get your head around the concepts and commands.

We start by creating our first empty Ubuntu container. The –i connects us to the shell of the container (interactive).

$ sudo docker run -i -t --name="firstcontainer" ubuntu:14.04 /bin/bash


Then in this case we need to install the python and web.py dependencies INSIDE of the container. This could be modified for any required dependencies or apps.

$ apt-get update
$ apt-get install -y python python-webpy


Within the container, create a new python script:

$ sudo mkdir /home/test1
$ sudo vi /home/test1/app.py


The contents of the script are:

#!/usr/bin/python
import web,sys
urls = (
 '/', 'index'
 )
app = web.application(urls, globals())
class index:
 def GET(self):
 argumentone = sys.argv[2]
 greeting = "Hello World, the test message is " + argumentone
 return greeting
if __name__ == '__main__' :
 app = web.application(urls, globals())
 app.run()


Exit the container, back to the Native OS:

$ exit


Confirm the name of your container (the last container run):

$ sudo docker ps –l
 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
 f711ff0fd695 ubuntu:14.04 /bin/bash 32 minutes ago Exit 0 firstcontainer


Create a new image from your docker called testpython1

$ sudo docker commit firstcontainer testpython:0.1


Confirm you can see the image and get the image ID:

$ sudo docker images
 REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
 testpython 0.1 fcb365f7591b 2 minutes ago 247.8 MB


Finally, start up 3 instances of your web application:

$ sudo docker run -d -p 8081:8081 fcb365f7591b python /home/test1/app.py 8081 "instance1"
$ sudo docker run -d -p 8082:8082 fcb365f7591b python /home/test1/app.py 8082 "instance2"
$ sudo docker run -d -p 8083:8083 fcb365f7591b python /home/test1/app.py 8083 "instance3"


Open a browser on your network and connect to http://dockerserverip:8081
Try the same for the other two port numbers. Note we now have a system running 3 separate containers which could then be load balanced using a third party tool, or even run completely different content. Cool huh?

Next, how to mount a drive into your container…

Looking Forward to Storage Field Day 9 (#SFD9)

Storage Field Day

So for those of you who love to nerd out on storage like I do, you have probably already heard of the awesome streaming events put on by Stephen Foskett and the crew from Tech Field Day, otherwise known as Storage Field Day. These have grown so popular that Stephen is having to put on extra events just to cater for demand, which I think speaks volumes as to their efficacy and indeed quality!

For those not yet indoctrinated, these events involve taking a group of around a dozen storage and technology delegates to visit a number of different startups (think Pure, NexGen, Coho, etc) and more established companies (think Intel!) to talk about the latest things going on both at those organisations and in the industry in general. Each session lasts a couple of hours, but is generally broken down into several bite sized chunks for consumption at your leisure.

As a stream viewer you get the opportunity to learn about your favourite vendors latest funky stuff and watch them answer questions about all the things you probably wanted to know but never got the chance to ask. It is also a great way to get your head around an unfamiliar technology or vendor. Lastly, if you watch live, you can also ask questions via twitter for the delegates to ask of the presenters.

As a delegate this goes to a whole new level as you get to spend almost an entire week mahoossively geeking out on tech, learning from some of the smartest people in the tech industry, and meeting with the senior people at some of the industry’s best-known companies. I find it generally safest just to wear multiple layers to avoid any embarrassing nerdgasms! 😉

So with that in mind I am really chuffed to have been invited back to attend Storage Field Day 9, next month (16th-18th March) in San Jose!

Not all of the companies have been announced as yet, but we already know that the likes of Cohesity, Intel, VMware & Violin Memory will be in attendance. More will be confirmed over the next coupe of weeks and having seen the provisional list I can tell you it is definitely going to be a great event!

vendors

Needless to say the lineup of delegates is awesome as usual, with many well known bloggers from the EU, US and APAC. Make sure you check them out and follow the crew on twitter if you are so inclined. Most delegates post their opinions around the vendors and tech both during and after the event, so make sure you check out their blog feeds. For example, here is mine:

http://www.tekhead.org/blog/feed/

If you want to tune in live, simply go to http://techfieldday.com from 16th-18th March (PST) or catch up with the recordings on youtube later.

Finally, be warned my Twitter stream does get rather busy during the event, so feel free to temporarily mute me if need be! 😉

%d bloggers like this: