Is “System” an Obsolete Concept?

I’ll cut to the chase: the short answer is “no.” But that comes with a really big caveat. In  reality, it’s more like “not entirely.”

We began developing software systems because in the beginning (around the late 1940s), there was no software. If you wanted to make a computer do something, you had to create the whole system: data input, storage, processing, output—all of it had to be created from the ground up. You couldn’t just create a new data analysis algorithm because there was nothing to install it into, no way of running it unless you built the whole system needed to get data to the algorithm and render the results to the user.

The world of software has changed since that time. Data input is readily available—every computer has a keyboard, every database has a query interface. Data storage is handled by either an off-the-shelf database product, or through disk read/write functions provided by the operating system. Many, if not most, of these functions, are available as services that can be invoked across the network. Data storage services like Amazon S3 make data storage just another external component that can be called as needed with no thought to how it is designed and implemented.

The proliferation of publicly-available services has led to the emergence of mashups—a new capability created by stringing together available services, adding a little glueware here and there, and presto! A new capability emerges, with 90% or more of that capability existing entirely outside the developer’s control. Thought of another way, we have a “system” that isn’t designed the way we traditionally think of a system: as a coherent set of components developed and assembled to perform a pre-defined set of functions. Traditional system architecture and engineering focused on developing the desired capability starting from the requirements and progressively building out the necessary infrastructure until the resulting assemblage could fulfill all those requirements. The result is a “system” that can be picked up and moved from one location to another with some minimal set of dependencies (e.g., a particular operating system as the installation location).

In a cloud environment, none of this makes sense. Most of the functions we think of as the infrastructure of a system (authentication, logging, data storage, etc.) area available as services. These are the things that often consumed the lion’s share of the engineering when developing a new system. Even user interface components are readily available as pre-packaged libraries, or even as third-party tools that will render any properly formatted data. In such an environment, are we even developing “systems” anymore? Yes, we are developing systems in the sense that we are defining the ways in which many different elements will be strung together to perform some function. But if the majority of those elements are outside our control—black boxes with interfaces we invoke as needed—are we really developing the whole system, or are we just developing some marginal increase in functionality in the context of a much larger environment? And does that marginal increase in functionality really require the heavyweight processes and frameworks that accompany traditional systems engineering?

No, we probably don’t need all those heavyweight processes and frameworks for much new capability development today. We need to find a new way to define and quantify how to make new capabilities available within a larger ecosystem, one that is light weight and focused on those marginal changes that are needed instead of documenting the vast corpus existing capability.

That said, there is still a place for the traditional concept of a system. Applications where a capability must operate by itself will still need those traditional processes and frameworks. Satellites, aircraft control systems, and similar applications that must be continue functioning even in the absence of any supporting elements will require a full-blown design process that fully thinks through all the elements needed to keep that system up and running. But that’s not the majority of software development these days.

No, “system” is not an –obsolete concept. But its usefulness is increasingly limited these days.