Is “System” an Obsolete Concept?

I’ll cut to the chase: the short answer is “no.” But that comes with a really big caveat. In  reality, it’s more like “not entirely.”

We began developing software systems because in the beginning (around the late 1940s), there was no software. If you wanted to make a computer do something, you had to create the whole system: data input, storage, processing, output—all of it had to be created from the ground up. You couldn’t just create a new data analysis algorithm because there was nothing to install it into, no way of running it unless you built the whole system needed to get data to the algorithm and render the results to the user.

The world of software has changed since that time. Data input is readily available—every computer has a keyboard, every database has a query interface. Data storage is handled by either an off-the-shelf database product, or through disk read/write functions provided by the operating system. Many, if not most, of these functions, are available as services that can be invoked across the network. Data storage services like Amazon S3 make data storage just another external component that can be called as needed with no thought to how it is designed and implemented.

The proliferation of publicly-available services has led to the emergence of mashups—a new capability created by stringing together available services, adding a little glueware here and there, and presto! A new capability emerges, with 90% or more of that capability existing entirely outside the developer’s control. Thought of another way, we have a “system” that isn’t designed the way we traditionally think of a system: as a coherent set of components developed and assembled to perform a pre-defined set of functions. Traditional system architecture and engineering focused on developing the desired capability starting from the requirements and progressively building out the necessary infrastructure until the resulting assemblage could fulfill all those requirements. The result is a “system” that can be picked up and moved from one location to another with some minimal set of dependencies (e.g., a particular operating system as the installation location).

In a cloud environment, none of this makes sense. Most of the functions we think of as the infrastructure of a system (authentication, logging, data storage, etc.) area available as services. These are the things that often consumed the lion’s share of the engineering when developing a new system. Even user interface components are readily available as pre-packaged libraries, or even as third-party tools that will render any properly formatted data. In such an environment, are we even developing “systems” anymore? Yes, we are developing systems in the sense that we are defining the ways in which many different elements will be strung together to perform some function. But if the majority of those elements are outside our control—black boxes with interfaces we invoke as needed—are we really developing the whole system, or are we just developing some marginal increase in functionality in the context of a much larger environment? And does that marginal increase in functionality really require the heavyweight processes and frameworks that accompany traditional systems engineering?

No, we probably don’t need all those heavyweight processes and frameworks for much new capability development today. We need to find a new way to define and quantify how to make new capabilities available within a larger ecosystem, one that is light weight and focused on those marginal changes that are needed instead of documenting the vast corpus existing capability.

That said, there is still a place for the traditional concept of a system. Applications where a capability must operate by itself will still need those traditional processes and frameworks. Satellites, aircraft control systems, and similar applications that must be continue functioning even in the absence of any supporting elements will require a full-blown design process that fully thinks through all the elements needed to keep that system up and running. But that’s not the majority of software development these days.

No, “system” is not an –obsolete concept. But its usefulness is increasingly limited these days.

What is a Capability?

If we are going to define a “capability-focused architecture,” it follows that we must define what, exactly, a “capability” is. The US Department of Defense defines a capability as “the ability to complete a task or execute a course of action under specified conditions and level of performance.” This is a more specific definition than that provided by most dictionaries, and in the arena of solution architecture, specificity is important.

Given that definition, how do we specify what a “capability” in terms that are meaningful enough to support a formal architecture model? To do that, we have to break down what a capability really is.

To start, let us take it as axiomatic that a capability may be composed of other, lower-level capabilities. For instance, the capability to drive a car is composed of the capabilities to start the car, to keep the car in the selected lane, to accelerate, to slow down, etc. So, any definition of “capability” must be recursive–there will always be some lower level of capability that could be decomposed. Given this, our primary tsk is to define how to describe the lowest level of capability that we desire to break down.

First, we must define what we mean by “task” or “course of action.” For the sake of simplicity, let us confine ourselves to defining a “task” at this time. Another way of thinking of a task is to effect a change in the state of the world. That is, there is some characteristic of the world that we can measure, and a change in the value of that measurement by some specified value indicates the task has been completed.

Second, we must define “specified conditions.” This is another way of saying “the current state of the world.” That is, the specified conditions are in reality a set of measurements of the characteristics that we care about. Completing a task means changing one or more of those values by some specified amount. “Specified conditions” means the initial measurements of those values that define the state of the world (obviously, limiting our definition of “world” to those characteristics that are meaningful to the problem at hand).

Finally, we must define “level of performance.” Level of performance has two aspects. The first aspect of level of performance is to define the acceptable rate of change of the characteristic of the state of the world which defines completion of the task (e.g., complete the task faster than some established time). The second aspect of level of performance is to define the acceptable amount of change to all the other characteristics of the state of the world (e.g., change characteristics X by amount Y white characteristic Z remains constant).

Defining a capability in this fashion provides the foundation, but by no means the full structure, for a capability-focused architecture. In fixture posts, I will try to flesh this idea out in greater detail.

 

Toward Capability-Focused Architecture

Traditional architecture frameworks are poorly suited to defining and managing an enterprise architecture. I think that assertion has been pretty thoroughly established. As I have written here and elsewhere, we need a new approach to enterprise architecture. But I have also stated that traditional architecture frameworks such as Zachmann, TOGAF, and DoDAF still have their place. But that place is not “everything except enterprise architecture.”

Traditional architecture frameworks originated in an age of stand-alone systems, and they are still best-suited to designing those systems. I don’t mean to say they cannot be used to design interconnected systems, but that they are best suited to systems that are intended for use where the system under development is largely intended to be deployed and operated as a single system. In most cases today, that means a system that is capable of operating disconnected from other systems (even if that means somewhat reduced functionality). Good examples of this type of system are flight control systems, satellite operating systems, or even applications such as word processors or spreadsheets.

The reality of software development today is the most systems are not the kind of old-fashioned stand-alone applications that traditional architecture frameworks were built for. In an age of cloud computing where mashups are common and building a new application on a framework like AWS or Facebook is the norm and not the exception, traditional architecture frameworks are not a good fit. They’re heavyweight and predisposed to systems where requirements can be clearly defined up front. In a DevOps environment, or in an enterprise where available interfaces from many providers can be composed into ne capabilities with a little glue ware, these frameworks struggle to deliver value that aligns with the amount of work they require.

The DevOps landscape requires a new kind of architecture framework, one based on defining capabilities instead of systems. Consider the case of building an application within a framework such as Facebook. Facebook provides some set of functions, such as authentication, that you do not need to develop. Instead, all you need to do is stitch together existing functions and add a little bit of glue ware to fill in those capabilities that are not explicitly available from the underlying platform.

This kind of development calls for a style of architecture that is based on defining capabilities based on the difference between the capabilities available vs. the capability desired. Traditional architecture frameworks are not capability-focused, which suggests they are not well-suited to this kind of work. What is lacking is a means of defining capabilities and what it takes to define a capability and whether it has been completed. I will attempt to better define that over the coming weeks.