Data Science is an emerging field that is plagued by lurid, often inconsequential reports of success. The press has been all too happy to predict the future demise of the human race.
But sifting through chaff, we do see some genuinely interesting reports of work that affects both bottom-line profit and top-line revenue.
Read moreCloud-Native, a collection of tools and best practices, disrupts the ideas behind traditional software development. I am a firm believer of the core concepts, which include visibility, repeatability, resiliency and robustness.
The idea begins in 2015 when the Linux Foundation formed the Cloud-Native Computing Foundation. The idea was to collect the tools and processes that are often employed to develop cloud-based software.
However, the result was a collection of best practices which extend well beyond the realms of the cloud. This post introduces the essential components: DevOps, continuous delivery, microservices and containers.
Read moreThe proliferation of Data Science is largely due to: ubiquitous data, increasing computational power and industry acceptance that solutions are an asset.
Data Science applications are no longer a simple dataset on a single laptop. In a recent project, we help develop a novel cloud-native machine learning service. It is unique in that problems are packaged as containers and submitted to the cloud for processing. This enables users to distribute and scale their models easily.
Read more
This presentation describes how Prometheus was integrated into a polyglot microservices application. We will use the “Sock Shop”, a cloud-native reference microservices architecture to demonstrate some of the best practices and pitfalls of attempting to unify monitoring in real life. Attendees will be able to use this application as a reference point, or as a real life starting point for their own applications.
Specifically, we will cover:
Last week I was working on a simple implementation updating a shopping cart for a site, the frontend was written in html/javascript. The brief - when the quantity of an item in the cart was modified the client could press an update cart button which would update the cart database, after which it was necessary to recalculate the total values of the order and refresh the page with the new totals.
Read moreThe terms “Cloud” or “Cloud Services” have become so laden with buzz that they would be happy to compete with Apollo 11 or Toy Story. But the hype often hides the most important aspects that you need to know. Like how it works, or what you can do with it. This is the first of several introductory pieces that focus on the very basics of modern applications.
Read moreIn one of my applications, for various reasons, we now have a batch like process and a HTTP based REST application running inside the same binary. Today I came up against an issue where HTTP latencies were around 10 seconds when the batch process was running.
After some debugging, the reason for this is that although the two are running in separate Go routines, the batch process is not allowing the scheduler to schedule the HTTP request until the batch process has finished.
Read moreGo introduced vendoring into version 1.5 of the language. The vendor folder is used as a dependency cache for a project. Because of the unique way Go handles dependencies, the cache is full code from an entire repository; worts and all. Go will search the vendor folder for its dependencies before it searches the global GOPATH.
Tools have emerged to corral the vendor folder and one of my favourites is glide. However, one thing that hasn’t happened yet is improvements to Go’s code coverage tool. As of 1.7 any attempt to generate a single coverage report for the entire project, excluding the vendor folder, is not possible. Single reports are important for code coverage services such as Coveralls.
Read moreThe testing of microservices is inherently more difficult than testing monoliths due to the distributed nature of the code under test. But distributed applications are worth pursuing because by definition they are decoupled and scalable.
With planning, the result is a pipeline that automatically ensures quality. The automated assurance of quality becomes increasingly important in larger projects, because no one person wants to or is able to ensure the quality of the application as a whole.
This article provides some guidelines that I have developed whilst working on a range of software with microservice architectures. I attempt to align the concepts with best practice (references at the end of this article), but some of the terminology is my own.
Read more