New Distributed Primitives for Developers

(This post was originally published on Red Hat Developers, the community to learn, code, and share faster. To read the original post, click here.)

Object-Oriented Primitives (in-process primitives)

As a Java developer, I'm well familiar with object-oriented concepts such as class, object, inheritance, encapsulation, polymorphism, etc. In addition to the object-oriented concepts, I'm also well familiar with the Java runtime, what features it provides, how I can tune it, how it manages my applications, what would be the lifecycle of my object and the application as a whole, etc.

And for over a decade, all that have been the primary tools, primitives and building blocks I've used a developer to create applications. In my mental model, I would use classes as components, which would give birth to objects that are managed by the JVM. But that model has started to change recently.

Kubernetes Primitives (distributed primitives)

In the last year, I began to run my Java applications on Kubernetes, and that introduced new concepts and tools for me to use. With Kubernetes I don't rely only on the object-oriented concepts and the JVM primitives to implement the whole application behavior. I still need to use the object-oriented building blocks to create the components of the application, but I can also use Kubernetes primitives for some of the application behavior.

For example, now I strive to organize the units of application behavior into independent container images which become the main building blocks. That allows me to use a new richer set of constructs provided by Kubernetes to implement the application behavior. For example, now I don't rely on only an implementation of ExecutorService to run some service periodically, but I can also use Kubernetes CronJob primitive to run my container periodically. The Kubernetes CronJob will provide similar temporal behavior, but use higher level constructs, and rely on the scheduler to do dynamic placement, performing health checks, and shutting down the container when the Job is done. All that ends up in more resilient execution with better resource utilization as a bonus. If I want to perform some application initialization logic, I could use the object constructor, but I could also use init-container in Kubernetes to carry out the initialization at a higher level.

The Distributed Mental Model

Having in-process primitives in the form of object-oriented concepts and the JVM features, combined with distributed out-fo-process primitives provided by Kubernetes give developers a richer set of tools to create better applications. When building a distributed application, my mental model is not any longer limited to a JVM, but spreads across a couple of nodes with multiple JVMs running in coordination.

The in-process primitives and the distributed primitives have commonalities, but they are not directly comparable and replaceable. They operate at different abstraction levels, have different preconditions and guarantees.  Some primitives are supposed to be used together, for example, we still have to use classes, to create objects and put them into container images. But some other primitives such as CronJob in Kubernetes can replace the ExecutorService behavior in Java completely. Here are few concepts which I find commonalities in the JVM and Kubernetes, but don't take that any further.

With time, new primitives give birth to new ways of solving problems, and some of these repetitive solutions become patterns. Check out my in-progress Kubernetes Patterns book for this line of thinking.

CloudNativeCon + KubeCon Europe 2017 Impressions

I was lucky to get my Cloud Native Patterns (video, slides) lightning talk accepted and attend CloudNativeCon + KubeCon Europe 2017 in Berlin. The following is a quick braindump / cameradump while the adrenaline and the excitement of the conference are still in my veins.

The conference had 1200 attendees which is 3x bigger than last year conference in London.

A few quick stats about Kubernetes community (video) by Chen Goldberg

What is Cloud Native and Why Should I Care (video)? by Alexis Richardson

The software is eating the world.
Open source is eating the software.
Cloud (is that Cloud Native?) is eating open source.

All sessions really well attended and packed and in some sessions people not let in. Below is shot from Autoscaling in Kubernetes (video) by Marcin Wielgus.

Also was interesting to see that Philips Hue (smart lights) started evaluating Kubernetes after last year's KubeCon and today they run in production all smart light backend.

A common theme across few sessions was about the fact that Kubernetes makes the life of Ops easy, but the life of the developers harder. The entry level for Kubernetes is quite high which prevents faster adoption.

Michelle Noorali from Deis did excellent talk on getting this point across, and so did Joe Beda.
Coming from a Java background, this is a topic that is close to my heart as well. I've been trying to educate the Java community why containerized Cloud Native and Kubernetes matter. And it is great to see that it is a widely recognized theme and a priority for the cloud native community.

Lot's of companies presented in the conference, from big players such Google, Red Hat, IBM and Microsoft (which also offer Kubernetes as a service), to Mesosphere. And many other smaller companies and new startups, where everybody does something around Cloud Native. (Would have been nice if Cloud Foundry had also shown up as the pioneers in Cloud Native).

Containerised USB sticks and Kubernerts based OpenShift books have all gone.

If you are looking to get involved into the cloud native world, check out the Job Board below for ideas and Red Hat jobs site as well.

Final thoughts:

  • At these events, you can see and feel how CNCF is building a great community of users accompanied by a collaborative ecosystem of companies.
  • At least half of the keynote sessions were given by women. That is at least 10x higher than other Open Source conferences.
  • Kubernetes (and other CNCF projects) have to become more user/developer friendly. Expect that to happen next!
  • All Recordings from the conferences are on youtube already. Check them out, feel the vibe and become part of it.
  • Don't miss CloudNativeCon + KubeCon December 6-8 2017 in Texas.

Microservices Hierarchy of Needs

This is a small extract from a longer post I published at The New Stack. Check the original post here.

Maslow's Hierarchy of Needs is a motivational theory in psychology, comprising of multitier model of human needs, often depicted as hierarchical levels within a pyramid. Maslow uses terms such as physiological, safety, belongingness and love, esteem, self-actualization, and self-transcendence to describe the stages that human motivation generally moves through.

I thought, I should apply it to Microservices too, as there is a clear list of needs that has to be satisfied in order to be successful in the Microservices journey. Once I listed the main Microservices concerns I couldn't stop myself noticing that Kubernetes does cover a big chunk of these needs pretty well. So I've added Kubernetes to the diagram as well and here is the result:

Microservices Hierarchy of Needs
Microservices Hierarchy of Needs

The key takeaway from this diagram is that Kubernetes can automate many of the boring Microservices related activities such as environment provisioning, resource management, application placement, deployment, health checks, restarts, service discovery, configuration management, auto scaling, etc. With all that taken care of by Kubernetes, the developers can focus on what really requires creativity and talent: analyzing the business problem, creating great domain driven designs, hidden behind beautiful APIs, with well crafted clean code, that is constantly refactored and adapting to change.

Recently, I've been blogging and speaking about why Kubernetes is the best platform for running Microservices styled applications. If you have not convinced yet, give it a try.

PS: I'll be talking at #CloudNativeCon + #KubeCon in Berlin about Cloud Native Design Patterns. Visit here to learn about my talk!

Microservices Deployments Evolution

(This post was originally published on Red Hat Developers, the community to learn, code, and share faster. To read the original post, click here.)

Microservices Are Here, to Stay

A few years back, most software systems had a monolithic architecture and slow release cycle. In the recent years, there is a clear move towards Microservices architecture which is optimized for scalability, elasticity, failure, and speed of change. This trend has been further enforced by the adoption of cloud and containers, which also enabled practices such as DevOps.
Trends in the IT Industry

All these changes have resulted in a growing number of services to develop and an even bigger number of deployments to do. It soon became clear that the explosion in the number of deployments cannot be controlled using pre-microservices tools and techniques, and new ways have been born. In this article, we will see how Cloud Native platforms such as Kubernetes allow deployment of Microservices in high scale with minimal human intervention. Here, I use Kubernetes as the example, but other container based Cloud Native platforms (Amazon ECS, Apache Mesos, Docker Swarm) do offer similar primitives and capabilities. In the not so distant future, the practices described here will become the common way for managing and deploying Microservices at scale.

Cloud Native Deployment Traits

Self-Service Environments

Local, Dev, Test, Int, Perf, Stage, Prod.... are all environments, but what is an environment really? Usually, it is a VM or a group of VMs that represent an environment. For example:
Local is the developer laptop where the user has full freedom to experiment and break stuff. It still has to be similar to other environments to avoid the "it works on my machine" syndrome.
Dev is the very first environment where changes from all developers are integrated into one working application. It runs SNAPSHOT version of the services, has mocked external dependencies, and for most of the time, it is broken from constant change.
Once a service has been released, it is moved to a more stable environment such as Test. This environment may be slightly more powerful (maybe have more than one VM), may have more external services available rather than mocks, and it has also testers accessing it.
While the environments get closer to production environment such as Int/Stage/Perf, they get bigger, with more VMs and more external dependencies available. At some point, we probably need an environment that has resources quantifiable with the production environment so we can do performance tests that mean something in relation to the production environment.
The main difficulty with this model is that the concept of environment is mapped to a physical or virtual VMs and as such, it is slow to alter. You cannot easily change the resource profile of an existing environment, create new environments, or give an environment per developer on the fly.
Environments managed by Kubernetes
In the Cloud Native world, an environment is just an isolated, controlled and named resource collection. For example, given a pool of 8 VMs, you can chunk and use that resource pool for different environment instances depending on the needs of the various teams. And those chunks don't map to VMs, which means it may be that multiple environments are collocated and share few of the VMs. Creating, editing, destroying an environment is achieved with one command, instantly, w/o a request for VMs and waiting for days or even weeks. This allows teams to change their environment profiles based on their changing needs in a self-service manner, easier and faster.

Dynamically Placed Applications

We can see how a self-service platform for managing environments can ease the onboarding of new developers, services, projects, and even enable custom release strategies which may require temporary environments on the fly. Once an environment is ready, the next task is choosing a strategy for placing our Microservices on the environments.
One of my favorite Microservices related books is Building Microservices by Sam Newman. In the book, Sam approaches Microservices from all possible angles and one of those is deployments. In the Deployment chapter, the author describes few strategies for mapping services to hosts and their pros and cons.

Service to Host Mapping Strategies
I have also described this approach in painful details for Apache Camel based applications in my Camel Design Patterns book. Basically, it comes down to choosing a way to package your services and grouping them on the available hosts considering all kind of service and host dependencies. Luckily for us, the industry has moved forward quickly and containers have been accepted as the standard for packaging Microservices. Unless you are Netflix and have over 100K Amazon EC2 instances, you shouldn't look for quick ways for burning EC2 images for each service, and instead just use containers. Even Netflix has moved on and they are experimenting with containers and even developing open source container scheduling software. So no more a VM per service, and no more service to host mapping strategies. Instead, based on your service requirements/dependencies and the available host resources, your Cloud Native platform should find a host for every service in a predictable manner defined by policies. That requires an understanding of each service and describing its requirements such as storage, CPU, memory, etc but later the benefits are huge. Rather than manually orchestrating and assigning each service to a host in advance, the Kubernetes scheduler performs a choreography of services and places them on the hosts dynamically when requested.
As you can see the concept of VM/Host disappears with environments spanning multiple hosts, and services being placed on hosts dynamically. We don't care and we don't want to know the actual hosts where our environments are carved and where the Microservices are placed (except when predictable performance is critical and shared host resources and platform resources should be avoided).

Declarative Service Deployments

We can provision environments in a self-service manner and have services placed on the environments with a minimal effort. But when we have multiple instances of a service, how do we deploy the new instances? Do we first have to stop an instance, then upgrade, and when things go wrong we rollback? Cloud Native platforms (and Kubernetes specifically) have thought about this too. Using the concept of Deployment Kubernestes allows describing how the service upgrades should be performed.
Rolling and Fixed Deployment

The Rolling Update strategy of the Deployment updates one pod at a time, rather than taking down the entire service at the same time and ensures zero downtime. This Fixed (or Recreate as it is named) strategy, on the other hand, brings downs all service instances, and then gradually starts new versions.
In both cases,  the Deployment will declaratively update the deployed application progressively behind the scene.

Additional Benefits

Having a platform that is capable of managing the full life cycle of services offers also additional deployment related benefits.

Blue-Green and Canary Releases

Blue-green release is a way for achieving rapid rollback in case anything goes wrong with the new release.
Canary release is a technique for reducing the risk of introducing a new software version in production by introducing change only to a small subset of users before rolling it out to everybody.
Blue-Green and Canary Releases
Both of these techniques can be easily achieved using Kubernetes with minimal human intervention.

Self Healing

The Cloud Native platform is able not only detect failure but also act and heal it. Kubernetes will regularly perform health checks for your application and if it detects something wrong, it will restart your service and even go further and move it to a different healthier host if required.

Auto Scaling

In addition to self healing, the platform is also capable of auto scaling of services and even the infrastructure. That is a very powerful feature giving the platform some Antifragile characteristics.

DevOps and Antifragile

If we look at all the benefits provided by Kubernetes, I think it is fair to say that it offers primitives and abstractions that are better suited for managing Microservices at scale. With such a tool, it becomes easier for teams and organizations to use practices such as DevOps and focus on improving the business processes towards Antifragility.

Closing Thoughts

Not a Free Lunch

We have seen many of the benefits of Cloud Native platforms in regards to Microservices deployments but have not discussed any of its cons in this article on purpose. As you may expect, there is a steep learning curve for Kubernetes, and also a need for a change in the patterns, principles, practices and processes when developing such applications. Basically, that is the move to Cloud Native applications.

Change is Inevitable

This may sound too strong, but if you are doing Microservices, and that is Microservices at scale, using a Cloud Native platform (such as Kubernetes) is a must.
Picture from Wikipedia. Wisdom from W. E. Deming
If you are using pre-microservices tools, techniques, and practices for developing Microservices, it will hit you back, and Microservices may not work for you.

Microservices are a Commodity

I'm a big fan of Simon Wardley. I'm not able to follow everything he writes about, but even the old writings are very interesting and somehow it all makes sense in retrospective. If you haven't read anything from him, this video is an excellent start (or go back a few years to the longer talk). In this article I'll try to make sense about what is happening in Microservices world using Wardly's theory (and diagrams).

How things evolve?

Any idea, product, system, etc., starts with its genesis, and if it is successful, it evolves, others copy it and create new custom solutions from it. If it is still successful, it diffuses further and others create new products which get improved, extended, and becomes widespread and available, “ubiquitous”, well understood and more of a commodity. This lifecycle can be observed in many successful products when looked over the years such as computers, mobiles, virtualization/cloud, etc.

If we think about the Microservices, the architectural style, the projects that support it, platforms that were born from it, containers, DevOps practice, etc... each of them is at a certain stage in the diagram above. But as a whole the Microservices movement now is a pretty widespread, well understood concept, and turning into commodity already. And there are many indications confirming that, starting from number of publications, conferences, books, confirmed success stories on production, etc. No doubt about it, not any longer.

How did we get here?

The Microservices genesis started 5-6 years ago with Fred George and James Lewis from ThoughtWorks sharing their ideas. In the next few months Thoughtworks did lot of thinking, writing, and talking about it, while Netflix did a lot of hacking and created the first generation of Microservices libraries. Most of those libraries were still not very popular, and usable by the wider developer community, and only the pioneers and startup minded companies would try them at times. Then SpringSource joined the bandwagon, they wrapped and packaged the Netflix libraries into products and made all the custom build solutions accessible and easy to consume for Java developers. In the meantime, all this interest in Microservices drove further innovation and containers were born. That brought in another wave of innovation, more funding, shuffling, new set of tools, which made the DevOps theory a practise.

Containers being the primary means for deploying Microservices, soon created the need for container orchestration i.e. Cloud Native platforms. And today, the Cloud Native landscape is in transition, taking its next shape. If you look around there are multiple Cloud Native platforms, each of which started its journey from different point in time and a unique value proposition , but slowly getting into a common feature set, similar concepts and even standards. For example the feature parity of platforms such as AWS ECS, Kubernetes, Apache Mesos, Cloud Foundry, are getting close, each being feature rich, used in production, and comparable primitives. As you can see from the diagram above, now what becomes important as technology strategy is to bet on platforms with open standards, open source, large community and high chance of long term success. That means for example choosing a container runtime that is OCI compliant, choosing a tracing tool that is based on Open Tracing standard rather than custom implementation, supporting the industry standard logging and monitoring solutions, supported by companies that are good in commodity products.

Organization Types

According to Wardly, there are three types of people/teams/organizations and each is good at certain stages of the evolutions:
  • Pioneers are good in exploring uncharted territories and undiscovered concepts. They turn into life the crazy ideas.
  • Settlers are good in turning the half baked prototype into something useful for a larger audience. They build trust, understanding and refine the concept. They turn the prototype into a product, make it manufacturable and turn it profitable.
  • Town Planners are good in taking something and industrialise it taking advantage of economies of scale. They build the trusted platforms of the future which requires immense skill. They find ways to make things faster, better, smaller, more efficient, more economic and good enough.

    With this definition and the above table showing the characteristics of each type of organisation, we can make the following hypothetical classification:
    • Netflix are definitely the Pioneers. The creative, path finder people they have, the way the company is set around experimentation, uncertainty, their culture around freedom, responsibility, everything they have brought into the microservice services world makes them pioneers.
    • SpringSource for me are more the settler type. They already had a popular Java stack, and they managed to spot the trend in Microservices, and created a good consumable product in the form of Spring Boot and Spring Cloud.
    • Amazon, Google, Microsoft are the town planners. They may come late, but they come well prepared, with the long term strategy defined, with web scale solutions and unbeatable pricing. Platforms such as Kubernetes, ECS (not entirely sure about the latter as it is pretty closed) are build on over 10 years of experience, and indented to last long, and become the undustry standard.
    One important takeway from this section is that, not everything invented by pioneers is meant for general consumption. Pioneers move fast, and unless your organisation has similar characteristics, it might be difficult to follow all the time. On the other hand, town planners create products and services which are interoperable and based on open standards. That in the longer term becomes an important axes of freedom. 


    In the Microservices world things are moving from uncharted to industrialised direction. Most of the activities are not that chaotic, uncertain and unpredictable. It is almost turning into a boring and dull activity to plan, design and implement Microservices. And since it is an industrialised job with a low margin, the choice of tool, and ability to swap those platforms plays a significant role.

    Last but not least, a nice side effect from this evolution is that we should hear less about Conway's Law, the two pizzas, and circuit breaker during conferences, and should hear more about managing Microservices at scale, automation, business value, serverless and new uncharted ideas from the pioneers in our industry.

    About Me