Today’s developers are expected to develop resilient and scalable
distributed systems. Systems that are easy to patch in the face of
security concerns and easy to do low-risk incremental upgrades. Systems
that benefit from software reuse and innovation of the open source
model. Achieving all of this for different languages, using a variety of
application frameworks with embedded libraries is not possible.
Recently I’ve blogged
about “Multi-Runtime Microservices Architecture” where I have explored
the needs of distributed systems such as lifecycle management, advanced
networking, resource binding, state abstraction and how these
abstractions have been changing over the years. I also spoke about “The
Evolution of Distributed Systems on Kubernetes” covering how Kubernetes
Operators and the sidecar model are acting as the primary innovation
mechanisms for delivering the same distributed system primitives.
On both occasions, the main takeaway is the prediction that the
progression of software application architectures on Kubernetes moves
towards the sidecar model managed by operators. Sidecars and operators
could become a mainstream software distribution and consumption model
and in some cases even replace software libraries and frameworks as we
are used to.
The sidecar model allows the composition of applications written in
different languages to deliver joint value, faster and without the
runtime coupling. Let’s see a few concrete examples of sidecars and
operators, and then we will explore how this new software composition
paradigm could impact us.
Out-of-Process Smarts on the Rise
In Kubernetes, a sidecar is one of the core design patterns
achieved easily by organizing multiple containers in a single Pod. The
Pod construct ensures that the containers are always placed on the same
node and can cooperate by interacting over networking, file system or
other IPC methods. And operators
allow the automation, management and integration of the sidecars with
the rest of the platform. The sidecars represent a language-agnostic,
scalable data plane offering distributed primitives to custom
applications. And the operators represent their centralized management
and control plane.
Let’s look at a few popular manifestations of the sidecar model.
Envoy
Service Meshes such as Istio, Consul, and others are using transparent service proxies such as Envoy
for delivering enhanced networking capabilities for distributed
systems. Envoy can improve security, it enables advanced traffic
management, improves resilience, adds deep monitoring and tracing
features. Not only that, it understands more and more Layer 7 protocols
such as Redis, MongoDB, MySQL and most recently Kafka. It also added
response caching capabilities and even WebAssembly support that will
enable all kinds of custom plugins. Envoy is an example of how a
transparent service proxy adds advanced networking capabilities to a
distributed system without including them into the runtime of the
distributed application components.
Skupper
In addition to the typical service mesh, there are also projects, such as Skupper,
that ship application networking capabilities through an external
agent. Skupper solves multicluster Kubernetes communication challenges
through a Layer 7 virtual network and offers advanced routing and
connectivity capabilities. But rather than embedding Skupper into the
business service runtime, it runs an instance per Kubernetes namespace
which acts as a shared sidecar.
Cloudstate
Cloudstate
is another example of the sidecar model, but this time for providing
stateful abstractions for the serverless development model. It offers
stateful primitives over GRPC for EventSourcing, CQRS, Pub/Sub,
Key/Value stores and other use cases. Again, it an example of sidecars
and operators in action but this time for the serverless programming
model.
Dapr
Dapr
is a relatively young project started by Microsoft, and it is also
using the sidecar model for providing developer-focused distributed
system primitives. Dapr offers abstractions for state management,
service invocation and fault handling, resource bindings, pub/sub,
distributed tracing and others. Even though there is some overlap in the
capabilities provided by Dapr and Service Mesh, both are very different
in nature. Envoy with Istio is injected and runs transparently from the
service and represents an operational tool. Dapr, on the other hand,
has to be called explicitly from the application runtime over HTTP or
gRPC and it is an explicit sidecar targeted for developers. It is a
library for distributed primitives that is distributed and consumed as a
sidecar, a model that may become very attractive for developers
consuming distributed capabilities.
Camel K
Apache Camel is a mature integration library that rediscovers itself on Kubernetes. Its subproject Camel K
uses heavily the operator model to improve the developer experience and
integrate deeply with the Kubernetes platform. While Camel K does not
rely on a sidecar, through its CLI and operator it is able to reuse the
same application container and execute any local code modification in a
remote Kubernetes cluster in less than a second. This is another example
of developer-targeted software consumption through the operator model.
More to Come
And these are only some of the pioneer projects exploring various
approaches through sidecars and operators. There is more work being done
to reduce the networking overhead introduced by container-based
distributed architectures such as the data plane development kit (DPDK),
which is a userspace application that bypasses the layers of the Linux
kernel networking stack and access directly to the network hardware.
There is work in the Kubernetes project to create sidecar containers with more granular lifecycle guarantees. There are new Java projects based on GraalVM implementation such as Quarkus
that reduce the resource consumption and application startup time which
makes more workloads attractive for sidecars. All of these innovations
will make the side-car model more attractive and enable the creation of
even more such projects.
Sidecars providing distributed systems primitives
I’d not be surprised to see projects coming up around more specific
use cases such as stateful orchestration of long-running processes such
as Business Process Model and Notation (BPMN) engines in sidecars. Job
schedulers in sidecars. Stateless integration engines i.e. Enterprise
Integration Patterns implementations in sidecars. Data abstractions and
data federation engines in sidecars. OAuth2/OpenID
proxy in sidecars. Scalable database connection pools for serverless
workloads in sidecars. Application networks as sidecars, etc. But why
would software vendors and developers switch to this model? Let’s see a
few of the benefits it provides.
Runtimes with Control Planes over Libraries
If you are a software vendor today, probably you have already
considered offering your software to potential users as an API or a
SaaS-based solution. This is the fastest software consumption model and a
no-brainer to offer, when possible. Depending on the nature of the
software you may be also distributing your software as a library or a
runtime framework. Maybe it is time to consider if it can be offered as a
container with an operator too. This mechanism of distributing software
and the resulting architecture has some very unique benefits that the
library mechanism cannot offer.
Supporting Polyglot Consumers
By offering libraries to be consumable through open protocols and
standards, you open them up for all programming languages. A library
that runs as a sidecar and consumable over HTTP, using a text format
such as JSON does not require any specific client runtime library. Even
when gRPC and Protobuf are used for low-latency and high-performance
interactions, it is still easier to generate such clients than including
third party custom libraries in the application runtime and implement
certain interfaces.
Application Architecture Agnostic
The explicit sidecar architecture (as opposed to the transparent one)
is a way of software capability consumption as a separate runtime
behind a developer-focused API. It is an orthogonal feature that can be
added to any application whether that is monolithic, microservices,
functions-based, actor-based or anything in between. It can sit next to a
monolith in a less dynamic environment, or next to every microservice
in a dynamic cloud-based environment. It is trivial to create sidecars
on Kubernetes, and doable on many other software orchestration platforms
too.
Tolerant to Release Impedance Mismatch
Business logic is always custom and developed in house. Distributed
system primitives are well-known commodity features, and consumed
off-the-shelf as either platform features or runtime libraries. You
might be consuming software for state abstractions, messaging clients,
networking resiliency and monitoring libraries, etc. from third-party
open source projects or companies. And these third party entities have
their release cycles, critical fixes, CVE patches that impact your
software release cycles too. When third party libraries are consumed as a
separate runtime (sidecar), the upgrade process is simpler as it is
behind an API and it is not coupled with your application runtime. The
release impedance mismatch between your team and the consumed 3rd party
libraries vendors becomes easier to manage.
Control Plane Included Mentality
When a feature is consumed as a library, it is included in your
application runtime and it becomes your responsibility to understand how
it works, how to configure, monitor, tune and upgrade. That is because
the language runtimes (such as the JVM) and the runtime frameworks (such
as Spring Boot or application servers) dictate how a third-party
library can be included, configured, monitored and upgraded.
When a software capability is consumed as a separate runtime (such as a
sidecar or standalone container) it comes with its own control plane in
the form of a Kubernetes operator.
That has a lot of benefits as the control plane understands the
software it manages (the operand) and comes with all the necessary
management intelligence that otherwise would be distributed as
documentation and best practices. What’s more, operators also integrate
deeply with Kubernetes and offer a unique blend of platform integration
and operand management intelligence out-of-the-box. Operators are
created by the same developers who are creating the operands, they
understand the internals of the containerized features and know how to
operate the best. Operators are executables SREs in containers, and the number of operators and their capabilities are increasing steadily with more operators and marketplaces coming up.
Software Distribution and Consumption in the Future
Software Distributed as Sidecars with Control Planes
Let’s say you are a software provider of a Java framework. You may
distribute it as an archive or a Maven artifact. Maybe you have gone a
step further and you distribute a container image. In either case, in
today’s cloud-native world, that is not good enough. The users still
have to know how to patch and upgrade a running application with zero
downtime. They have to know what to backup and restore its state. They
have to know how to configure their monitoring and alerting thresholds.
They have to know how to detect and recover from complex failures. They
have to know how to tune an application based on the current load
profile.
In all of these and similar scenarios, intelligent control planes in
the form of Kubernetes operators are the answer. An operator
encapsulates platform and domain knowledge of an application in a
declaratively configured component to manage the workload.
Sidecars and operators could become a mainstream software
distribution and consumption model and in some cases even replace
software libraries and frameworks as we are used to.
Let’s assume that you are providing a software library that is
included in the consumer applications as a dependency. Maybe it is the
client-side library of the backend framework described above. If it is
in Java, for example, you may have certified it to run it on a JEE
server, provided Spring Boot Starters, Builders, Factories, and other
implementations that are all hidden behind a clean Java interface. You
may have even backported it to .Net too.
With Kubernetes operators and sidecars all of that is hidden from the
consumer. The factory classes are replaced by the operator, and the
only configuration interface is a YAML file for the custom resource. The
operator is then responsible for configuring the software and the
platform so that users can consume it as an explicit sidecar, or a
transparent proxy. In all cases, your application is available for
consumption over remote API and fully integrated with the platform
features and even other dependent operators. Let’s see how that happens.
Software Consumed over Remote APIs Rather than Embedded Libraries
One way to think about sidecars is similar to the composition over inheritance principle
in OOP, but in a polyglot context. It is a different way of organizing
the application responsibilities by composing capabilities from
different processes rather than including them into a single application
runtime as dependencies. When you consume software as a library, you
instantiate a class, call its methods by passing some value objects.
When you consume it as an out-of-process capability, you access a local
process. In this model, methods are replaced with APIs, in-process
methods invocation with HTTP or gRPC invocations, and value objects with
something like CloudEvents. This is a change from application servers
to Kubernetes as the distributed runtime. A change from
language-specific interfaces, to remote APIs. From in-memory calls to
HTTP, from value objects to CloudEvents, etc.
This requires software providers to distribute containers and
controllers to operate them. To create IDEs that are capable of building
and debugging multiple runtime services locally. CLIs for quickly
deploying code changes into Kubernetes and configuring the control
planes. Compilers that can decide what to compile in a custom
application runtime, what capabilities to consume from a sidecar and
what from the orchestration platform.
Software consumers and providers ecosystem
In the longer term, this will lead to the consolidation of
standardized APIs that are used for the consumption of common primitives
in sidecars. Rather than language-specific standards and APIs we will
have polyglot APIs. For example, rather than Java Database Connectivity
(JDBC) API, caching API for Java (JCache), Java Persistence API (JPA),
we will have polyglot APIs over HTTP using something like CloudEvents.
Sidecar centric APIs for messaging, caching, reliable networking, cron
jobs and timer scheduling, resource bindings (connectors to other APIs,
protocols), idempotency, SAGAs, etc. And all of these capabilities will
be delivered with the management layer included in the form of operators
and even wrapped with self-service UIs. The operators are key enablers
here as they will make this even more distributed architecture easy to
manage and self-operate on Kubernetes. The management interface of the
operator is defined by the CustomResourceDefinition and represents
another public-facing API that remains application-specific.
This is a big shift in mentality to a different way of distributing
and consuming software, driven by the speed of delivery and operability.
It is a shift from a single runtime to multi runtime application
architectures. It is a shift similar to what the hardware industry had
to go through from single-core to multicore platforms when Moore’s law
ended. It is a shift that is slowly happening by building all the
elements of the puzzle: we have uniformly adopted and standardized
containers, we have a de facto standard for orchestration through
Kubernetes, possibly improved sidecars coming soon, rapid operators
adoption, CloudEvents as a widely agreed standard, light runtimes such
as Quarkus, etc. With the foundation in place, applications,
productivity tools, practices, standardized APIs, and ecosystem will
come too.
This post was originally published at The New Stack here.