Logistics in the Age of Cyber Warfare

Posted on by

It’s an old military truism that amateurs worry about tactics, and professionals worry about logistics. The origin of the saying is attributed to remarks made by Gen Robert H. Barrow, Commandant of the Marine Corps, in 1980: “Amateurs talk about tactics, but professionals study logistics.”

The point is that supplying a military force is the hard part of operations, and takes  significant effort and expertise, while nearly anyone can learn the basics of tactics. The best tactics are of little use to an army that does not have the food, fuel, or ammunition to carry out the plan. The Allies won WWII because they could supply more aircraft, tanks, ammunition, and fuel to their forces, as well as keeping them fed.

Logistics is the art and science of overcoming terrain. Moving people and materiel means traversing the rivers, swamps, oceans, and mountains from the source of supply to the point where they are needed. If an army is unable to traverse the terrain, then resupplying their forces is impossible.

In the realm of cyber warfare, many make the mistake of thinking that logistics is not a significant part of the effort, but that is a mistaken view. Connectivity and bandwidth are the logistics of the cyber domain. To engage in cyber offense or defense, one must get to the portion of the network where the fight is taking place. The best cyber warriors in the world are of no use if they cannot connect to the network they are tasked with attacking or defending. Connectivity to the area of conflict, and sufficient bandwidth to deliver the intended payload, is essential to succeeding in cyber war.

By the same token, denying one’s adversary connectivity to resources that one wishes to attack or defend is critical to success. In physical warfare, interdicting an enemy’s supply lines is a key objective in most campaigns. If the enemy cannot resupply their forces, they will eventually be worn down and defeated just as the Germans were at Stalingrad in the winter of 1942-43. The German army was an excellent fighting force, but without food, fuel, and munitions they were doomed.

So when considering cyber warfare, remember that connectivity and bandwidth are not incidental considerations–they are what the professionals are worried about.

Reconsidering Data

Posted on by

Master data management has firmly embedded itself into the collective leadership psyche. Well, the terminology has. But whether decision-makers really understand what they should be managing is another question entirely.

I spend a lot of time in data meetings, usually involving government organizations. The majority of the time is spent worrying about “data tagging.” The discussions focus on what should be tagged, which tags to use, etc. Some of these discussions have been going on for fifteen years or more. I’ve seen the same argument over the same issue run on for over five years with no decision. That’s just a waste of time; there’s no other word for it.

The problem is that we’re trying to manage the wrong things, and we’re doing that because of some unexamined assumptions. Understanding that, and thinking clearly about possible solutions, will require understanding a little bit about how we got here.

In 1994, the Dublin Core Metadata Initiative (DCMI) was established to define metadata standards for documents on the World Wide Web. In a nutshell, their goal was to define an XML version of a library card catalog so that I could be electronically cataloged, simplifying search and retrieval of documents.

Dublin Core was a good idea for its time. But its time has passed. Technology has passed it by, but too many data managers have not noticed. Think about it: library catalogs date as far back as 700 BC, and the modern card catalog as we used to know it dates to 1780.  Works were indexed by subject, author, and title. This made sense in the days when it was impossible to index the entirety of each work.

Modern technology has made the old-fashioned card catalog obsolete, replaced by an electronic catalog. By the same token, the catalog structure itself is obsolete due to the march of technology. The card catalog was devised because it was impractical to create detailed indices of an entire work. The solution was to mine out specific search terms; title, subject, author, and important keywords in the text. Even into the early 2000s this made sense; scanning and indexing an entire work was impractical for large bodies of work. That is no longer the case. A document of a hundred pages can be fully indexed in less time than it takes to write this sentence. for documents that are reasonably well-structured, it is even feasible to identify the author and title.

And yet we persist in managing document metadata as if we were still beholden to a physical card catalog. Why?

It would be better to toss out, or drastically strip down, standards like Dublin Core and replace them with a more comprehensive approach that assumes the entire contents of a work will be fully indexed and electronically searchable (which is the current reality, whether we want to recognize it or not). Continuing to treat document metadata as something unique is the source of endless problems.

In reality, digital data only comes in one of two forms: as a binary object (a picture, a Word document, etc.) or as tuples. All the text inside a document can be treated as one or more tuples for purposes of managing data. Even if it is stored as a unit (i.e., a file containing text), tuples that describe that body of text can be easily mined out of it.

Instead of managing a card catalog standard, we should be focused on managing tuples.

 

 

 

Architecture is Ephemeral

Posted on by

Before I begin, I must confess that this is not my original idea. That distinction goes to Gerhard Beck, a friend and colleague whose disregard of pieties in the enterprise architecture field has provoked a great deal of thought and argument (in the best sense of that word).

The 19th century military theorist Carl von Clausewitz asserted that no plan survives first contact with the enemy. By the same token, no architecture survives first contact with the developers. Events intervene–requirements change, components do not work as anticipated, new capabilities come to market–all these things collude to make the as-built system vary from the as-designed system to one degree or another. And that does not even take into account how rarely developers actually consult the architecture while constructing a system. And no one goes back to update the architecture document to reflect the as-built system. That kind of update costs money and is not seen as a value-added activity by program managers who are trying to keep costs and schedules under control.

So systems never reflect the architecture. The moment development begins, the architecture is a historical artifact. The only architecture that matters is the one on the network. And that one is not static; it is always changing.

Traditional architecture frameworks do not recognize this fundamental fact. All traditional architecture frameworks derive from the original Zachman framework. Conceived in a time when system development projects were major, expensive undertakings, the Zachman framework does not suit all modern architecture needs as well as it once did. It’s a fine framework for new-start systems, but it just is not responsive enough to rapidly changing technology and business needs for the modern IT ecosystem. In an age of Agile and DevOps, traditional architecture frameworks are at a disadvantage.

So, if traditional architecture frameworks are not helpful in understanding the operation of a modern, dynamic enterprise, what is?

Most large enterprises have deployed host-based monitoring agents as part of their cybersecurity strategy. These agents gather a wealth of important architectural data and forward it to a central server. Where it is devoutly ignored by the cybersecurity experts who are trying to protect the enterprise. Do not get me wrong–that is not a criticism of the cyberwarriors out there. I am merely pointing out that they have other fish to fry. There is a wealth of data about the real. operational architecture that can be reused to understand what is really happening in the enterprise. Understanding that ecosystem is where enterprise architects need to focus their efforts.

What is a Task?

Posted on by

In order to evolve from a view of architecture focused on systems to one focused on capabilities, we need to define what a “capability” is. And as I’ve mentioned, that means defining what a “task” is. After all, if a capability is the ability to complete a task, then it follows that we cannot define “capability” until we define “task.”

A task is a representation of the change in the state of the world. Put another way, a task is a potential change in some set of values that measure the state of the world. To complete a task is to change those values–that is, to change the state of the world. For example, if the task is to sharpen a pencil, then we can say that the current state of the world (i.e., the pencil is not sharp) is represented by the angle formed by the point of the pencil, say 45 degrees. We can define the task “sharpen the pencil” as a change in the angle formed by the point of the pencil to some value less than 45 degrees. If we change the angle formed by the point of the pencil to 40 degrees, then we have completed the task of sharpening the pencil.

Stated more generally, we can define a task as a change in some set of values measured at time t1 and time t2. This gives us the foundation for formally defining a capability, but as a capability is defined as being able to complete a tasks under specified conditions, it follows that we must also define what we mean by “specified conditions.” At least some of those conditions are restrictions on the task itself. Possible conditions include:

  • Thresholds for each value as measured at t1 and t2
  • The time elapsed between t1 and t2
  • The rate of change in any value over the interval t1 to t2

There are many other possibilities for “specified conditions” which I will explore in more detail as time permits.

Is “System” an Obsolete Concept?

Posted on by

I’ll cut to the chase: the short answer is “no.” But that comes with a really big caveat. In  reality, it’s more like “not entirely.”

We began developing software systems because in the beginning (around the late 1940s), there was no software. If you wanted to make a computer do something, you had to create the whole system: data input, storage, processing, output—all of it had to be created from the ground up. You couldn’t just create a new data analysis algorithm because there was nothing to install it into, no way of running it unless you built the whole system needed to get data to the algorithm and render the results to the user.

The world of software has changed since that time. Data input is readily available—every computer has a keyboard, every database has a query interface. Data storage is handled by either an off-the-shelf database product, or through disk read/write functions provided by the operating system. Many, if not most, of these functions, are available as services that can be invoked across the network. Data storage services like Amazon S3 make data storage just another external component that can be called as needed with no thought to how it is designed and implemented.

The proliferation of publicly-available services has led to the emergence of mashups—a new capability created by stringing together available services, adding a little glueware here and there, and presto! A new capability emerges, with 90% or more of that capability existing entirely outside the developer’s control. Thought of another way, we have a “system” that isn’t designed the way we traditionally think of a system: as a coherent set of components developed and assembled to perform a pre-defined set of functions. Traditional system architecture and engineering focused on developing the desired capability starting from the requirements and progressively building out the necessary infrastructure until the resulting assemblage could fulfill all those requirements. The result is a “system” that can be picked up and moved from one location to another with some minimal set of dependencies (e.g., a particular operating system as the installation location).

In a cloud environment, none of this makes sense. Most of the functions we think of as the infrastructure of a system (authentication, logging, data storage, etc.) area available as services. These are the things that often consumed the lion’s share of the engineering when developing a new system. Even user interface components are readily available as pre-packaged libraries, or even as third-party tools that will render any properly formatted data. In such an environment, are we even developing “systems” anymore? Yes, we are developing systems in the sense that we are defining the ways in which many different elements will be strung together to perform some function. But if the majority of those elements are outside our control—black boxes with interfaces we invoke as needed—are we really developing the whole system, or are we just developing some marginal increase in functionality in the context of a much larger environment? And does that marginal increase in functionality really require the heavyweight processes and frameworks that accompany traditional systems engineering?

No, we probably don’t need all those heavyweight processes and frameworks for much new capability development today. We need to find a new way to define and quantify how to make new capabilities available within a larger ecosystem, one that is light weight and focused on those marginal changes that are needed instead of documenting the vast corpus existing capability.

That said, there is still a place for the traditional concept of a system. Applications where a capability must operate by itself will still need those traditional processes and frameworks. Satellites, aircraft control systems, and similar applications that must be continue functioning even in the absence of any supporting elements will require a full-blown design process that fully thinks through all the elements needed to keep that system up and running. But that’s not the majority of software development these days.

No, “system” is not an –obsolete concept. But its usefulness is increasingly limited these days.

What is a Capability?

Posted on by

If we are going to define a “capability-focused architecture,” it follows that we must define what, exactly, a “capability” is. The US Department of Defense defines a capability as “the ability to complete a task or execute a course of action under specified conditions and level of performance.” This is a more specific definition than that provided by most dictionaries, and in the arena of solution architecture, specificity is important.

Given that definition, how do we specify what a “capability” in terms that are meaningful enough to support a formal architecture model? To do that, we have to break down what a capability really is.

To start, let us take it as axiomatic that a capability may be composed of other, lower-level capabilities. For instance, the capability to drive a car is composed of the capabilities to start the car, to keep the car in the selected lane, to accelerate, to slow down, etc. So, any definition of “capability” must be recursive–there will always be some lower level of capability that could be decomposed. Given this, our primary tsk is to define how to describe the lowest level of capability that we desire to break down.

First, we must define what we mean by “task” or “course of action.” For the sake of simplicity, let us confine ourselves to defining a “task” at this time. Another way of thinking of a task is to effect a change in the state of the world. That is, there is some characteristic of the world that we can measure, and a change in the value of that measurement by some specified value indicates the task has been completed.

Second, we must define “specified conditions.” This is another way of saying “the current state of the world.” That is, the specified conditions are in reality a set of measurements of the characteristics that we care about. Completing a task means changing one or more of those values by some specified amount. “Specified conditions” means the initial measurements of those values that define the state of the world (obviously, limiting our definition of “world” to those characteristics that are meaningful to the problem at hand).

Finally, we must define “level of performance.” Level of performance has two aspects. The first aspect of level of performance is to define the acceptable rate of change of the characteristic of the state of the world which defines completion of the task (e.g., complete the task faster than some established time). The second aspect of level of performance is to define the acceptable amount of change to all the other characteristics of the state of the world (e.g., change characteristics X by amount Y white characteristic Z remains constant).

Defining a capability in this fashion provides the foundation, but by no means the full structure, for a capability-focused architecture. In fixture posts, I will try to flesh this idea out in greater detail.

 

Toward Capability-Focused Architecture

Posted on by

Traditional architecture frameworks are poorly suited to defining and managing an enterprise architecture. I think that assertion has been pretty thoroughly established. As I have written here and elsewhere, we need a new approach to enterprise architecture. But I have also stated that traditional architecture frameworks such as Zachmann, TOGAF, and DoDAF still have their place. But that place is not “everything except enterprise architecture.”

Traditional architecture frameworks originated in an age of stand-alone systems, and they are still best-suited to designing those systems. I don’t mean to say they cannot be used to design interconnected systems, but that they are best suited to systems that are intended for use where the system under development is largely intended to be deployed and operated as a single system. In most cases today, that means a system that is capable of operating disconnected from other systems (even if that means somewhat reduced functionality). Good examples of this type of system are flight control systems, satellite operating systems, or even applications such as word processors or spreadsheets.

The reality of software development today is the most systems are not the kind of old-fashioned stand-alone applications that traditional architecture frameworks were built for. In an age of cloud computing where mashups are common and building a new application on a framework like AWS or Facebook is the norm and not the exception, traditional architecture frameworks are not a good fit. They’re heavyweight and predisposed to systems where requirements can be clearly defined up front. In a DevOps environment, or in an enterprise where available interfaces from many providers can be composed into ne capabilities with a little glue ware, these frameworks struggle to deliver value that aligns with the amount of work they require.

The DevOps landscape requires a new kind of architecture framework, one based on defining capabilities instead of systems. Consider the case of building an application within a framework such as Facebook. Facebook provides some set of functions, such as authentication, that you do not need to develop. Instead, all you need to do is stitch together existing functions and add a little bit of glue ware to fill in those capabilities that are not explicitly available from the underlying platform.

This kind of development calls for a style of architecture that is based on defining capabilities based on the difference between the capabilities available vs. the capability desired. Traditional architecture frameworks are not capability-focused, which suggests they are not well-suited to this kind of work. What is lacking is a means of defining capabilities and what it takes to define a capability and whether it has been completed. I will attempt to better define that over the coming weeks.

Systems or Capabilities?

Posted on by

Architecture itself is not as broken as I had begun to think. The issue is not so much that architecture is broken, it is that the way we do development has evolved considerably, while our conception of architecture has not.

Architecture frameworks rest on the assumption that all problems should be approached from the same point of view: that the desired functionality must be created from the ground up. At the time most of those frameworks were developed, that was true. In the 1990s (about the time most architecture frameworks were developed), if you wanted to create a system to correlate photos to places on a map, you had a lot of development work to do. You would need to create or reuse a software module that displays maps. You would need a database to store the information correlating individual pictures to their place on the map, and you would probably want to standardize the kind of location designation (latitude/longitude, Universal Transverse Mercator, etc.) If you wanted a multi-use capability, you would need to create some means for managing user identities. You likely would have to create every capability from the ground up. If there were some libraries you could reuse, you would have to ensure they were written in a compatible language and operating system. If you were creating an application to run on Microsoft Windows, you couldn’t use a mapping package written for Unix. Furthermore, networking capabilities were variable if they were available at all. Ethernet, Token Ring, Fiber Distributed Data Interface, and other networking protocols were still competing for dominance. Any networking within the application would have to make some decisions about how to access the network and which protocols to support. And a network connection was by no means assured; most people who had a network connection were using a dial-up connection through their phone line.

Current development practices really don’t have to deal with so many complexities. There are libraries for almost any conceivable function for nearly any language and operating system. With networking standardized on Ethernet and the ubiquity of broadband connections, many capabilities can just be invoked over the network using web services. Need a map? Take your pick from Google, Bing, or many others. Need a user management capability? Facebook, Google, and a number of other companies will gladly handle your authentication and authorization needs.

What this all means for the developer is that creating a new capability no longer means creating a new system. Often, all that is necessary is to stitch together existing capabilities plus a few new modules or some glue ware and the new capability is ready. In many (if not most) cases, all of this runs over a network with the various capabilities hosted in a number of places and invoked over the network as needed. The user’s operating system is increasingly becoming nearly irrelevant with the move to cloud-hosted applications. This makes most of a traditional architecture framework either irrelevant; much of what isn’t irrelevant is overkill and is too heavyweight to be efficient when using modern development practices. This is even more obvious when employing DevOps practices.

That said, there are some exceptions. Systems that must operate in isolation, without network access or with very constrained bandwidth, often have to be built the old-fashioned way. Every function needed must be hosted locally, so thinking though all of the functions needed and how they will be accessed remains an important task. Examples of such systems include tactical military systems, aircraft flight control systems, and the systems used for interplanetary space missions. Put another way, there are some cases where we still need to develop entire systems. Traditional architecture frameworks serve these development efforts very well.

So I draw the conclusion that the problem with traditional architecture frameworks is that they only solve one problem well: the problem of soup-to-nuts development that really only applies to systems that must operate in isolation. These frameworks are inefficient for use when developing a new capability by stitching together existing capabilities with glue ware.

In short, what we need is a solution architecture framework that is tailored to developing capabilities instead of systems.

How Badly is Architecture Broken?

Posted on by

I do most of my work with the Department of Defense, so naturally I am most familiar with events in that arena. What I am seeing lately leads me to wonder if it is not must enterprise architecture that needs to be rethought, but perhaps architecture in general needs to be rethought.

Formal information systems architecture can trace its roots back to the development of the Zachman Architecture Framework in 1987. At the time Zachman developed his framework, we were still inventing the idea of information systems. Since we have borrowed so many terms from the building trades, I’ll use that context to draw a crude analogy: Developing information systems in 1987 was like building a house before the 2×4 stud had been invented. Everything was a custom-crafted, one-off solution. There was very little reuse across information systems.

Reuse of packaged components didn’t become a real possibility until the development of object-oriented programming. Object-oriented programming first emerged with the Simula 67 language, but Simula was tailored to the modeling and simulation community. General-purpose object-oriented programming first became practical with the release of the C++ language in 1985 (only two years before the Zachman framework was published).

Reusing capabilities that are already resident on the network was no more than a dream in 1987. A rudimentary version of the Internet existed, but the World Wide Web wouldn’t be invented for another two years. Simple Object Access Protocol (SOAP), the foundation of the first generation of web services, was first published in 1999, and the Representation State Transfer (REST) protocol was published in 2000.

Given all these changes, do we even need system architecture frameworks anymore, or are they anachronisms–relics of a bygone era and of no more use than a buggy whip? Are they still needed, but only in limited cases, such as when building a stand-alone system or capability as a service?

Thinking About Enterprise Architecture

Posted on by

For a number of years now I’ve been the lead architect for what can reasonably be called a system-of-systems interoperability effort across a portion of the US Department of Defense (DoD). It’s not an integration effort because there is no intent to combine the systems or formally link them together. The intention has been more to enable them to reuse each other’s components and / or to exchange data as needed.

To enable this, we created an enterprise reference architecture. We defined it using the DoD Architecture Framework (DoDAF). We created it, reviewed it with stakeholders, updated it based on their feedback, and published it. We’ve updated it roughly quarterly for nearly a decade, publishing and sharing it with all who were interested. And then we graded the various programs that fell within its purview on how well they conformed to the guidelines in the reference architecture. We even published an annual report on their progress to senior leadership could gauge the effectiveness of the effort.

And it failed. The affected systems do not interoperate any better than they did a decade ago. They don’t reuse components to any significant degree. And they largely ignore the reference architecture in the process.

There are a number of reasons for this failure. Many of those reasons can be put down to perverse incentives in the DoD procurement system; others can be attributed to the fact that the office publishing the reference architecture has no budgetary authority over the programs it is trying to govern. But there is one reason that is even more fundamental: the discipline of enterprise architecture as it is currently practiced is based on flawed assumptions.

I am far from the first person to conclude that enterprise architecture is not delivering on its promises. People have been pointing out that enterprise architecture has failed for over a decade. More recently, others have declared that enterprise architecture is dead. But for all of the articles I have read that are in a similar vein, most just dismiss the entire field as either a failure or obsolete in the face of changes in the software development process such as Agile and DevOps. There is a more compelling argument to be made that the problem with enterprise architecture is that we’ve been doing it wrong, such as the argument made here.

I have come to the conclusion that the problem is not with the idea of enterprise architecture, the problem is with the execution of enterprise architecture projects and the frameworks that are used by most enterprise architecture projects. Toward that end I have been devising a new way of doing enterprise architecture. More to follow in the coming weeks and months.