Cloud Native

How to Test Terraform Infrastructure Code

Published

Aug 22, 2018

Author

Dr. Phil Winder

CEO

Infrastructure as code has become a paradigm, but infrastructure scripts are often written and run only once. This works for simplistic infrastructure requirements (e.g. k8s deployments). But when there is a requirement for more varied infrastructure or greater resiliency then testing infrastructure code becomes a requirement. This blog post introduces a current project that has found tools and patterns to deal with this problem.

Cloud Native Data Science: Best Practices

Published

May 4, 2018

Author

Dr. Phil Winder

CEO

Following the Cloud Native best practices of immutability, automation and provenance will serve you well in a CNDS project. But working with data brings its own subtle challenges around these themes.

Cloud Native Data Science: Technology

Published

May 3, 2018

Author

Dr. Phil Winder

CEO

Technology choices in data-driven products are, as you would expect, largely directed by the type and amount of data. The first and most crucial decision to make is whether the data will be processed in a batch or streaming fashion.

Cloud Native Data Science: Strategy

Published

May 2, 2018

Author

Dr. Phil Winder

CEO

Data Science has become an important part of any business because it provides a competitive advantage. Very early on, Amazon’s data on book purchases allowed them to deliver personalised recommendations whilst customers were browsing their site. Their main competitor in the US at the time was Borders, who mainly operated in physical stores. This physicality prevented them from seamlessly providing customers with personalised recommendations [1]. This example highlights how strategic business decisions and data science are inextricably linked.

How to List all AMIs for each region in AWS

Published

Apr 3, 2018

Author

Dr. Phil Winder

CEO

A current project required a list of Amazon Machine Images (AMIs) for all regions for use in terraform. I couldn’t find a script to do this for me, so here you will find one that uses the aws cli, jq and a bit of Bash.

Introduction to Monitoring Microservices with Prometheus

Published

Dec 4, 2017

Author

Dr. Phil Winder

CEO

https://prometheus.io is an open source time series database that focuses on capturing measurements and exposing them via an API. I love Prometheus because it it so simple; it’s minimalism is its greatest feature. It achieves this by pulling metrics from instrumented applications, not pulling like many of its competitors. In other words Prometheus “scrapes” the metrics from the application.

This means that it works very well in a distributed, cloud-native environment. All of the services are unburdened by load on the monitoring system. This has knock on effects meaning that HA is supported through simple duplication and scaling is supported through segmentation.

Logging vs Tracing vs Monitoring

Published

Nov 30, 2017

Author

Dr. Phil Winder

CEO

What do you mean by monitoring? Why do you need it? What are the real needs and are you monitoring them? Ask yourself these questions. Can you answer them? If not, you’re probably doing monitoring wrong.

This post asks the basic question. What is monitoring? How does it compare to logging and tracing? Let’s find out.

What is Cloud-Native?

Published

Jun 21, 2017

Author

Dr. Phil Winder

CEO

Cloud-Native, a collection of tools and best practices, disrupts the ideas behind traditional software development. I am a firm believer of the core concepts, which include visibility, repeatability, resiliency and robustness.

The idea begins in 2015 when the Linux Foundation formed the Cloud-Native Computing Foundation. The idea was to collect the tools and processes that are often employed to develop cloud-based software.

However, the result was a collection of best practices which extend well beyond the realms of the cloud. This post introduces the essential components: DevOps, continuous delivery, microservices and containers.

Cloud-Native Data Science: Turning Data-Oriented Business Problems Into Scalable Solutions

Published

Jun 12, 2017

Author

Abstract The proliferation of Data Science is largely due to: ubiquitous data, increasing computational power and industry acceptance that solutions are an asset. Data Science applications are no longer a simple dataset on a single laptop. In a recent project, we help develop a novel cloud-native machine learning service. It is unique in that problems are packaged as containers and submitted to the cloud for processing. This enables users to distribute and scale their models easily.

Monitor My Socks: Using Prometheus in a Polyglot Open Source Microservices Reference Architecture

Published

Apr 29, 2017

Author

Abstract This presentation describes how Prometheus was integrated into a polyglot microservices application. We will use the “Sock Shop”, a cloud-native reference microservices architecture to demonstrate some of the best practices and pitfalls of attempting to unify monitoring in real life. Attendees will be able to use this application as a reference point, or as a real life starting point for their own applications. Specifically, we will cover: Integrating Prometheus in Java/Go/Node.