The value of software architecture

A favorite hobby of IT architects is to discuss the purpose of IT architecture and what value it contributes to an organisation. See for example The Three C's of Architecture, or the motto of the last Nationaal Architectuur Congres in the Netherlands: Architecture múst contribute (as if that were not obvious!).

The goal of information technology is to improve business processes using computers. Improve in the sense of speeding up the processes, reducing their cost or increasing the quality of their results. But information technology is of course not the only way to do this: processes can also be simplified or streamlined, or eliminated entirely if it appears that no one is waiting for their results. All without any automation in sight. This step is often skipped by the way. This process improvement is the area of business architects and enterprise architects, depending on the definition of these terms.

Only when software is used (purchased, constructed), software architecture comes into play. There are lots of definitions of software architecture of course. My favorite is that of Jan Dietz, quoted and explained in: Architecture, a definition by Johan den Haan:

Theoretically, architecture is the normative restriction of design freedom. Practically, architecture is a consistent and coherent set of design principles.

The beauty of this definition, I think, is that it clearly distinguishes between the concepts of architecture and design, which are mixed up in most other definitions. A good reason to restrict design freedom is that it saves on having to think. Thinking takes time and energy, is difficult and error-prone. So thinking should be avoided wherever possible, to be saved for the problems for which it is really needed!

By the above definition, software architecture focuses on the design of software and thus on improving part of the process of developing software. And, like in the case of business processes above, improving in the sense of speeding it up, reducing its cost or increasing the quality of its result. And, again like with business processes, there are more ways to do this besides software architecture, like for instance, development methods and tools.

Just like information technology must deliver value to the business processes, software architecture must deliver value to the process of developing software. In other words: the goal is the same - to deliver value - but the thing to which that value is delivered is different: software development instead of business processes. If you look at it this way, the value of software architecture for the organisation as a whole is at best indirect. For that value is only added after the enhanced software development process has delivered more value to the organisation, at lower cost, or in less time. This idea should make software architects humble.

How agile is architecture?

Ever since software is being developed, one of the greatest challenges is how to build exactly what the customer needs. How do you stay close to your customer in the face of forces that drive you away from him? Many methods and programming languages have promised a solution to this problem.

Take COBOL, one of the oldest programming languages. COBOL means COmmon Business Oriented Languages. The idea was that COBOL was business oriented enough so that business people could write their own programs with it. This never worked of course. COBOL is widely regarded as a difficult programming language. Good COBOL programmers are hard to find and they are badly needed to maintain the still enormous body of COBOL programs.

SQL: same story. In SQL you tell the computer, or more specifically the database, what you want with the data instead of how to do it. Should be easy enough for business people? Nowadays SQL is considered too difficult even for the average programmer. We have Object-Relational Mappers to shield programmers from the "complexity" of SQL.

Next candidate: UML. We draw diagrams, pictures that business people can understand or even draw themselves. We have a code generator that translates the pictures to programs, and the user is in control again. This approach, however recent, is already out of favour again. See my earlier post "What's new about Model-Driven Development". The use of DSL's instead of UML does help a little, because a DSL is simpler. But the expectation of some, that the domain specific nature of a DSL means that a domain expert can write programs with it, can safely be regarded as naïve. If only because the domain of a DSL is almost never the domain of the user, but that of the developer.

OK, so new programming languages or tools don't help to bridge the distance between user and programmer. Let's try something completely different. Let's have architects help the business user to describe his business processes, and align the applications to support those. And let's introduce a Service-Oriented Architecture (SOA), so the applications can communicate easily via services. And we need a process engine to directly support the business processes by calling the services in the correct order. "A business user can do this himself."

Sounds great! Only I don't believe a word of it. Firstly, SOA experts keep telling us that a SOA is difficult to implement, and that the potential benefits - easy adaptability of the software to the ever changing requirements of the business - will be only reaped after many years. That is, if the SOA is introduced properly and embedded in the organisation. Governance is the magic word here. So what do you mean: "quick adaptation to changing business requirements"? Secondly, the word governance with me evokes the idea of "programming by committee". And that should be faster than one or two programmers sitting together? Thirdly, now we have architects between the customer and the programmer, making the distance between them greater again. Not to say anything bad about the good ones (I know one or two), but many architects have little affinity with the practical problems ("irrelevant details") facing programmers trying to implement their SOA. I always hear architects complain about programmers not keeping to their guiding principles. Why would that be? And does it sound strange that those organisations that have a separate architecture department are the same whose development process has all but come to an almost complete standstill?

Is there no hope then? Yes, there is. Those projects that adopt a radical agile process, be it eXtreme Programming, Scrum, Evo or whatever, are the same that see a radical improvement in productivity, while actually building what the customer wants. In addition to the fact that the stories sound convincing, I have experienced this myself. That experience was so good that since then I don't want, and maybe even can't, use any other process.

Even though the results are good, there is still much resistance against agile methods. One objection raised sometimes is, that large systems cannot be built without architecture, and that agile methods lead to chaotic and unmanageable systems. The first is true: large systems still need architecture, but the second is not: agile does not necessarily lead to unmanageable chaos. Agile is no substitute for good craftmanship. To create excellent applications in an agile way you need excellent programmers. Only the way architecture is created is completely different. Architecture is a byproduct of software development. It emerges at the same time, not before. Only then you may hope to have an architecture that fits the application like a tailor-made suit.

Applications are so complex nowadays, that they can't be built without architecture. The same complexity is the reason why the architecture can't be thought out completely in advance. If you try you will end up with a rigid system that is hard to change, exactly what that same architecture is trying to avoid. It doesn't have to change, it is thought out in advance. No. Architecture is needed, but can only be developed via an agile process.