Archive for 28th November 2016

Scale-Out. Distributed. Whatever the Name, it’s the Future of Computing

Scale Out

We are currently living in the fastest period of innovation in the technology space which there has probably ever been. New companies spring up every week with new ideas, some good, some bad, some just plain awesome and unexpected!

One of the most common trends I have seen in this however was described in a book I read recently, “The Second Machine Age” by Erik Brynjolfsson & Andrew McAfee. This trend is that the majority of new ideas are (more often than not) unique recombinations of old ones.

Take for example the iPhone. It was not the first smart phone. It was not the first mobile phone, the first touch screen, or the first device to run installable apps. However, Apple recombined an existing set of technologies into a very compelling product.

We also reached a point a while back where clock speeds of CPUs are no longer increasing, and even CPUs are scaling horizontally. Workloads are therefore typically being designed to scale horizontally instead of vertically, taking advantage of the increased compute resources available whilst avoid being locked to vertically scaling clock speeds.

Finally, another trend we have seen in the industry of late is inexpensive and low power CPUs from ARM, being used in all sorts of weird and wonderful places; often providing solutions to problems we didn’t even know we had. Up until now, their place has generally been confined outside of the data centre. I am, however, aware of a number of companies now working on bringing them to the enterprise in a big way!

So, in this context of recombination, imagine then if you could provide a scale-out storage architecture where every single spindle had its own compute directly attached. Then combine many of these “nano-servers” together in a scale-out JBOD form factor on subscription pricing, all managed from a Meraki-style cloud portal… well that’s exactly what Igneous Systems have designed!

Igneous Systems Nano-Servers

One of the coolest things about scaling out like this, is that instead of a small number of large fault domains based around controllers, you actually end up with many tiny fault domains instead. The loss of any one controller or drive is basically negligible within the system and replacements can be sorted at the convenience of the administrators, rather than panicking about replacement of components asap. Igneous claim that you can also scale fairly linearly, avoiding the traditional bottlenecks of a dual controller (or similar) system. It will be interesting to see some performance benchmarks as they become available!

It’s still early days, so they are doing code deployments at some pretty high rates, around every 2 weeks, and to be honest I think there is a bit of work to be done around clarity of their SLAs, but in general it looks like a very interesting platform, particularly when pricing is claimed to be as low as half the price of Amazon S3.

Now as you might expect from a massively distributed solution, the entry point is not small, typically procured in 212TiB chunks, so don’t expect to use it for your SMB home drives! If however you have petabyte-scale data volumes and are looking for an on-prem(ises!) S3 compatible datastore, then its certainly worth looking at Igneous.

The future in the scale-out space is certainly bright, now if only I could get people to refactor their single-threaded applications!… 🙂

Further Info

You can catch the full Igneous session at the link below – it certainly was unexpected and interesting, for sure!

Igneous Systems Presents at Tech Field Day 12

Further Reading

Some of the other TFD delegates had their own takes on the presentation we saw. Check them out here:

Disclaimer: My flights, accommodation, meals, etc at Tech Field Day 12 were provided by Tech Field Day, but there was no expectation or request for me to write about any of the vendors products or services.

AWS Certified Developer Associate (CDA) Exam Experience & Tips

The information bellow covers my experience for the AWS Certified Developer Associate (CDA) exam from Amazon. Following this I will post a list of my study materials, so keep checking back for updates or check out my Index of AWS Posts.

Before you continue reading, I would first just say that this is my second AWS exam, having completed the AWS Certified Solution Architect Associate exam earlier this year. As such the materials I used to study towards the exam are more sparse, due to the level of knowledge I already have.

For a really full picture of all of the materials I’ve used over the last 12 months, I highly recommend you check out the Certified Solution Architect Associate exam experience and the Certified Solution Architect Associate study guides, along with a number of tips, tricks and gotchas I have posted over the past few months. I also did a podcast recently with Scott Lowe on the subject of learning AWS. If you are new to AWS, I highly recommend you check it out!

AWS Certified Developer Associate CDA Full Stack Journey

Certified Developer Associate Exam Experience

My personal experience of the AWS Certified Developer Associate exam was that it was quite a bit easier than the Solution Architect Associate exam. Now, I don’t know whether this is more because I have been doing quite a bit of AWS work, as well as writing about it quite a bit in the months since I passed the CSA, or if this was down to the exam being genuinely easier. Most likely a combination of the two, as many people seem to rate the three exams as Developer, Solutions Architect, then SysOps Associate in increasing difficulty.

Either way the exam itself was actually very reasonable if you have any experience working with AWS. The way that AWS seem to structure their exams, is with some general questions across their portfolio, then specific technologies taking precedence in each. The developer exam was no different; there is definitely a distinct bias towards DynamoDB, S3, SQS and authentication. All the things which AWS Developers are likely to use when building distributed and highly scalable applications of course!

It is worth noting that AWS do not expect you to be a developer to pass the exam. You don’t need to know how to code in any language or similar. It would be useful for you to understand the basic format of JSON, but again this isn’t critical to pass the exam. If you want to work hands on with any of the AWS tooling in real life however, this is pretty critical!

The exam itself is 80 minutes and 55 questions. Again AWS (as is their way) do some odd things like not giving you a passing grade requirement, but it’s generally safe to assume that if you get 70% or more, in the Certified Developer Associate exam then you will pass. The Kryterion exam environment is frankly a little poor / dated, but I already wrote about that in the CSA guide here, so I won’t repeat myself again! Suffice to say, read the other article for a detailed overview.

There’s not a huge amount of advice I can give regarding the exam itself, other than if you are stuck, go with your gut. Believe it or not, the most obvious answer is often the actual answer! Don’t second guess yourself and say “No way it couldn’t be that simple!”.AWS Certified Developer Associate CDA Gut Feeling

Things like the specific API syntax used by AWS is generally quite logical, however there are a few weird things! For example, the read commands GetItem and BatchGetItem match syntax with eachother and are logical, but the write equivalents, PutItem and BatchWriteItem, do not! Knowing these types of little weird things can potentially help you come exam time so make sure you memorise some of the more common API calls.

It is also very worthwhile practicing your DynamoDB maths, as AWS expect you to be able to do this in your head. Memorising and practicing Ryan’s simple method really helped me to get my head around it.

Finally, if there is one thing I recommend you read, it’s the DynamoDB FAQ. This is a goldmine of information that will stand you in good stead for both the exam, and developing solutions on AWS!

Best of luck, and if you found this article useful, please leave a comment below! 🙂

Want to Learn More?

For part 2 of this article, the AWS Certified Developer Associate exam study guide and materials, see here:

AWS Certified Developer Associate (CDA) Exam Study Guide and Materials

%d bloggers like this: