Along with many others, I recently created a Mastodon account, to be ready if and when Twitter becomes unhealthy. Creating this blogpost on this site I haven't used in over a decade is just a convenient way to have a link back to my Mastodon account, which is the only requirement for verification.
2022-11-10
2011-09-25
The daily team development checklist
Have you ever seen software development team members around five in the afternoon commit their changes for the day into the source code management system and go home, happily whistling, just before the Continuous Integration build finishes with errors that they introduced? And you had to stay and correct their errors before you could commit your own changes and go home yourself? I have seen this more than once and have devised a checklist to help avoid it.
It also has a tendency to promote quality, because it makes it transparent who causes the build to fail. Social pressure does the rest.
The checklist is based on the premise that the code base on the main line of development should always be stable, compiling and working correctly, as demonstrated by a set of automated tests that succeed one hundred percent. Most of the items in the list may seem self-evident, but every single one of them I have seen violated again and again. Not following the list mostly has the effect of leaving the more tedious and unpleasant parts of the work of developing software as a team to others.
- Checkout the newest version of the code base from the source code repository.
It is useless to build on stale code. Sooner or later your work will have to be integrated with the work of your team members, and the longer you wait, the more difficult it is. - Build the code and run all automated tests.
This step is to make sure that you start with clean code and that all errors that you will encounter later on are the results of your own changes. If you know the cause of errors, they are much easier to find. Use the same automated build procedure that the Continuous Integration build uses. That is the one that is controlled by version management and is the same for every developer. Usually this will be a set of ant or maven (or make) scripts. Especially do not rely on Eclipse building your code, because the result will be dependent on which (versions of) plugins you have installed and other team members do not, and on your settings and preferences.
If the build fails before you have changed anything, try to find out who introduced the errors, almost always the last one to commit code, and ask him to correct them. If that person is unavailable (see the first paragraph of this post), there is nothing left but to try to correct the errors yourself, while trying to suppress your anger. - Edit/compile/test to implement your features.
This is what is traditionally is seen as the main part of your work, which of course it is. Do not make too many changes at once, but try to break them up into working sub-features. The more changes you commit at once, the more difficult it is for your teammates to integrate them with theirs. And don't forget to write automated tests. You will see in the next step why this is in your own best interest, in addition to being necessary to maintain a high level of software quality. - Build the code and run all automated tests.
This step is the same as step 2, but this time it is executed to make sure that (1) you do not introduce faulty code, and (2) your changes do not break existing features. Features that have no automated tests run a risk of being accidentally broken or removed. I have seen developers who refused to write automated tests, because they found that 'too much work'. When they protested that their features were broken or disappeared in later versions, the legitimate reply was: 'if you had written tests, we would have known when we broke your features and kept them intact, so don't whine'. - Get hold of the semaphore.
Find the object that is used as a semaphore and put it on your desk next to your keyboard.
This step is explained later when we come to the last step. - Pull new changes from the source code repository.
Quite probably your teammates have been committing changes while your were busy developing your features, and you are the one responsible for integrating them with yours. Most source code management systems can automatically merge the new changes with yours and that is a useful feature. Sometimes this results in conflicts where you changed the same lines in the same file as others. Of course you have to resolve these, but the logical conflicts are usually more subtle. The version management system cannot detect those, but hopefully they are brought to light by failing automated tests. Therefore: - Build the code and run all automated tests.Again the same as step 1, but now with the purpose of making sure that your changes work well with those of other developers. Resolving logical conflicts may involve talking to other developers or the Product Owner (if you use Scrum) or analyst to find out what is the intended behavior. This is why many developers do not like this step. Apart from having to actually talk to people, it means you have to leave your own private train of thought and try to understand others. And worse, try to understand other developers code!
- Commit your changes.
Now, if all tests succeed, you may finally commit your changes, of course with associated remarks explaining the purpose of your changes. And with the id's referring to the issues in your issue tracking system that you addressed with your changes, if that is the policy in your team. Especially do not forget to add new files that you created. When this checklist is followed and when a Continuous Integration build is in place, forgetting to add new files to the source code repository remains the most common cause of failed builds. - Release the semaphore.Put the object used as a semaphore back to its neutral territory.
When using this checklist, it is possible that several developers keep themselves busy in an endless cycle of commits. The scenario that could lead to this is as follows: developer number one executes step 6 of this checklist and starts integrating the changes with his own. But before he is ready for step 8, developer number two has committed his changes first. When those changes occur in the same files that developer number one has changes, most source code management systems will warn him and refuse to commit the changes. Another more dangerous possibility is that the other changes are in different files, in which case the source code management system is not aware of the possible logical conflict. In this case, the result of the commit may be a broken build, exactly the thing that we are trying to avoid all the time. Of course you can pull any new changes again, but if you have bad luck, and the other developers are very productive, you could be busy like this forever.
This scenario can be avoided by introducing a semaphore, for example a physical object that can be put on your desk next to your computer. The associated rule is that only one developer at a time, the one that has the semaphore on his desk, is allowed to commit code. The other ones have to wait until the semaphore is released.
In the past I have used objects that make sounds, like a horn with a rubber ball or a hotel bell. In that case an extra sub step is introduced just before step 9: Sound the semaphore, to celebrate the progress. And of course to notify other developers that the semaphore is free again.
It also has a tendency to promote quality, because it makes it transparent who causes the build to fail. Social pressure does the rest.
2011-02-28
Question: how to do acceptance testing in Scrum?
When starting to use Scrum for software development, many teams approach acceptance testing in a more or less traditional way. Developers focus on implementing the spring backlog items within the sprint. At the end of the sprint a new release of the system is delivered and declared ready for acceptance testing.
There are is a problem with this approach. When a Product Backlog Item is not tested for acceptance, is is not completely Done. When acceptance testing is done after the sprint, that is, during the next sprint, any corrections resulting from its feedback can only be planned in the sprint after that. Much too late! This approach tends to result in an ever increasing stack of corrections taking time that should be used to implement new Product Backlog Items, and thus slowing down development.
So, why don't we do acceptance testing within the sprint? In this way we receive earlier feedback and have the opportunity to repair any issues right away and have the Backlog Item really done by the end of the sprint. But this approach has its own problems. For instance, team members cannot do acceptance testing themselves, but someone outside has to do this. This means that the software must be made available to some acceptance testing environment, where outsiders can perform their tests. They need a reasonable amount of time for that, which means that near the end of the sprint developers have to stop developing because whatever they deliver cannot be tested anymore. That means loss of precious time.
Do you have experience with acceptance testing within Scrum sprints? By the way, this problem exists in every iterative style of development; not just in Scrum. What solutions work for you?
There are is a problem with this approach. When a Product Backlog Item is not tested for acceptance, is is not completely Done. When acceptance testing is done after the sprint, that is, during the next sprint, any corrections resulting from its feedback can only be planned in the sprint after that. Much too late! This approach tends to result in an ever increasing stack of corrections taking time that should be used to implement new Product Backlog Items, and thus slowing down development.
So, why don't we do acceptance testing within the sprint? In this way we receive earlier feedback and have the opportunity to repair any issues right away and have the Backlog Item really done by the end of the sprint. But this approach has its own problems. For instance, team members cannot do acceptance testing themselves, but someone outside has to do this. This means that the software must be made available to some acceptance testing environment, where outsiders can perform their tests. They need a reasonable amount of time for that, which means that near the end of the sprint developers have to stop developing because whatever they deliver cannot be tested anymore. That means loss of precious time.
Do you have experience with acceptance testing within Scrum sprints? By the way, this problem exists in every iterative style of development; not just in Scrum. What solutions work for you?
2011-02-10
There's no such thing as platform-independence.
The central model in the Model-Driven Architecture (MDA) is the Platform-independent Model (PIM). The functionality of a system is specified it in a way that is independent of the platform used to implement it. To realize this implementation, the PIM is transformed into one or more platform-specific Models (PSM), from which source code is generated in a programming language like C, Java or .Net. That of course still needs to be compiled into machine code or byte code, but that is so obvious that it is usually not even mentioned.
The vision of MDA is attractive: to isolate the specification of system from continuous change in current implementation platforms. The idea is that if they change, you just have to develop new transformations to PSM's for the new platform, and new code generators. The specification of the system as a PIM remains unchanged.
One widely used programming languages used as a target for code generation, is Java. That's funny, because Java itself is already platform-independent. The C programming language actually is platform-independent too, but only at the source level, so that it has to be recompiled for any combination of the underlying operating system and processor architecture. But Java itself is also a platform for which alternatives exist, like .Net/C#, Scala or Ruby.
What has become of the vision of the MDA? In any case, it has not become a resounding success. Otherwise we would now all be working with MDA. In practice, there are many practical problems associated with this approach, making it less ideal than it seemed at the time. Partly because MDA relies heavily on UML, which is not designed, and not very suitable, for code generation. The approach that resembles the MDA the most is based on Domain-Specific Languages (DSL), but the differences are significant. If ten years ago you had specified all your systems as PIM’s in UML, they would now be pretty worthless. In theory, of course you could write a transformation into a platform that is now common, but that is not as easy as it seems.
In fact, the MDA with UML is also just a platform. And a platform independent-model is not fundamentally different from a program. Only the language is different. And that is the fate of any programming or specification language that is presented as platform-independent in the beginning. Eventually, even when the platform is successful, or perhaps especially then, alternatives will be developed, just like C# was developed as an alternative to Java. And then we will need another platform-independent platform from which code for these various alternatives can be generated. And so the circle is closed. Platform-independence is a moving target, an illusion, a utopia. There is no such thing!
The vision of MDA is attractive: to isolate the specification of system from continuous change in current implementation platforms. The idea is that if they change, you just have to develop new transformations to PSM's for the new platform, and new code generators. The specification of the system as a PIM remains unchanged.
One widely used programming languages used as a target for code generation, is Java. That's funny, because Java itself is already platform-independent. The C programming language actually is platform-independent too, but only at the source level, so that it has to be recompiled for any combination of the underlying operating system and processor architecture. But Java itself is also a platform for which alternatives exist, like .Net/C#, Scala or Ruby.
What has become of the vision of the MDA? In any case, it has not become a resounding success. Otherwise we would now all be working with MDA. In practice, there are many practical problems associated with this approach, making it less ideal than it seemed at the time. Partly because MDA relies heavily on UML, which is not designed, and not very suitable, for code generation. The approach that resembles the MDA the most is based on Domain-Specific Languages (DSL), but the differences are significant. If ten years ago you had specified all your systems as PIM’s in UML, they would now be pretty worthless. In theory, of course you could write a transformation into a platform that is now common, but that is not as easy as it seems.
In fact, the MDA with UML is also just a platform. And a platform independent-model is not fundamentally different from a program. Only the language is different. And that is the fate of any programming or specification language that is presented as platform-independent in the beginning. Eventually, even when the platform is successful, or perhaps especially then, alternatives will be developed, just like C# was developed as an alternative to Java. And then we will need another platform-independent platform from which code for these various alternatives can be generated. And so the circle is closed. Platform-independence is a moving target, an illusion, a utopia. There is no such thing!
2010-11-10
A false claim about loose coupling
Sometimes when loose coupling between modules is described as a desirable property (which it is of course), an somewhat exaggerated claim is made about its benefits. See for example the section "The Benefits" in OSGi – The Business Drivers:
Take for example a module that offers a service to transfer money from a checking account to a saving acount. Its interface will probably specify the account numbers of the accounts involved and the amount of money and the currency as required parameters. What if a new implementation, instead of faithfully depositing the exact same amount to the saving account that it withdraws from the checking account, now does nothing at all? Or worse, deposits just a little less that the amount withdrawn? Or performs the transfer of money in the reverse direction? Does this not break the 'wider system'? Of course it does!
Coupling may be more subtle than just whether or not it breaks compilation. Testing the new configuration is still needed, automated (preferably) or manual. And testing is usually still an order of magnitude more expensive than compilation.
As long as the module boundaries don’t change, one can change the functionality of the module freely, without concern for breaking the wider system; i.e. the impact of any local change is prevented from leaking into the wider system.When the implementation of a module is changed without changing its interface, the only certainty that you have is that it can be invoked just like its previous implementation, without recompilation. But above that, nothing is certain. My point is that a module is invoked for a reason: to perform some useful work. The interface of a module says something about the structure of the information that it needs to do its work, but very little about the work that it will do, and very little about how the information provided to it will be used to influence the work done.
Take for example a module that offers a service to transfer money from a checking account to a saving acount. Its interface will probably specify the account numbers of the accounts involved and the amount of money and the currency as required parameters. What if a new implementation, instead of faithfully depositing the exact same amount to the saving account that it withdraws from the checking account, now does nothing at all? Or worse, deposits just a little less that the amount withdrawn? Or performs the transfer of money in the reverse direction? Does this not break the 'wider system'? Of course it does!
Coupling may be more subtle than just whether or not it breaks compilation. Testing the new configuration is still needed, automated (preferably) or manual. And testing is usually still an order of magnitude more expensive than compilation.
2010-09-04
The value of software architecture
A favorite hobby of IT architects is to discuss the purpose of IT architecture and what value it contributes to an organisation. See for example The Three C's of Architecture, or the motto of the last Nationaal Architectuur Congres in the Netherlands: Architecture múst contribute (as if that were not obvious!).
The goal of information technology is to improve business processes using computers. Improve in the sense of speeding up the processes, reducing their cost or increasing the quality of their results. But information technology is of course not the only way to do this: processes can also be simplified or streamlined, or eliminated entirely if it appears that no one is waiting for their results. All without any automation in sight. This step is often skipped by the way. This process improvement is the area of business architects and enterprise architects, depending on the definition of these terms.
Only when software is used (purchased, constructed), software architecture comes into play. There are lots of definitions of software architecture of course. My favorite is that of Jan Dietz, quoted and explained in: Architecture, a definition by Johan den Haan:
The beauty of this definition, I think, is that it clearly distinguishes between the concepts of architecture and design, which are mixed up in most other definitions. A good reason to restrict design freedom is that it saves on having to think. Thinking takes time and energy, is difficult and error-prone. So thinking should be avoided wherever possible, to be saved for the problems for which it is really needed!
By the above definition, software architecture focuses on the design of software and thus on improving part of the process of developing software. And, like in the case of business processes above, improving in the sense of speeding it up, reducing its cost or increasing the quality of its result. And, again like with business processes, there are more ways to do this besides software architecture, like for instance, development methods and tools.
Just like information technology must deliver value to the business processes, software architecture must deliver value to the process of developing software. In other words: the goal is the same - to deliver value - but the thing to which that value is delivered is different: software development instead of business processes. If you look at it this way, the value of software architecture for the organisation as a whole is at best indirect. For that value is only added after the enhanced software development process has delivered more value to the organisation, at lower cost, or in less time. This idea should make software architects humble.
The goal of information technology is to improve business processes using computers. Improve in the sense of speeding up the processes, reducing their cost or increasing the quality of their results. But information technology is of course not the only way to do this: processes can also be simplified or streamlined, or eliminated entirely if it appears that no one is waiting for their results. All without any automation in sight. This step is often skipped by the way. This process improvement is the area of business architects and enterprise architects, depending on the definition of these terms.
Only when software is used (purchased, constructed), software architecture comes into play. There are lots of definitions of software architecture of course. My favorite is that of Jan Dietz, quoted and explained in: Architecture, a definition by Johan den Haan:
Theoretically, architecture is the normative restriction of design freedom. Practically, architecture is a consistent and coherent set of design principles.
The beauty of this definition, I think, is that it clearly distinguishes between the concepts of architecture and design, which are mixed up in most other definitions. A good reason to restrict design freedom is that it saves on having to think. Thinking takes time and energy, is difficult and error-prone. So thinking should be avoided wherever possible, to be saved for the problems for which it is really needed!
By the above definition, software architecture focuses on the design of software and thus on improving part of the process of developing software. And, like in the case of business processes above, improving in the sense of speeding it up, reducing its cost or increasing the quality of its result. And, again like with business processes, there are more ways to do this besides software architecture, like for instance, development methods and tools.
Just like information technology must deliver value to the business processes, software architecture must deliver value to the process of developing software. In other words: the goal is the same - to deliver value - but the thing to which that value is delivered is different: software development instead of business processes. If you look at it this way, the value of software architecture for the organisation as a whole is at best indirect. For that value is only added after the enhanced software development process has delivered more value to the organisation, at lower cost, or in less time. This idea should make software architects humble.
2009-08-21
Model Driven Development with Mod4J
My article "Model Driven Development met Mod4J" (in Dutch) appeared in the June 2009 issue of Java Magazine. Actually my first published article!
2009-03-20
How agile is architecture?
Ever since software is being developed, one of the greatest challenges is how to build exactly what the customer needs. How do you stay close to your customer in the face of forces that drive you away from him? Many methods and programming languages have promised a solution to this problem.
Take COBOL, one of the oldest programming languages. COBOL means COmmon Business Oriented Languages. The idea was that COBOL was business oriented enough so that business people could write their own programs with it. This never worked of course. COBOL is widely regarded as a difficult programming language. Good COBOL programmers are hard to find and they are badly needed to maintain the still enormous body of COBOL programs.
SQL: same story. In SQL you tell the computer, or more specifically the database, what you want with the data instead of how to do it. Should be easy enough for business people? Nowadays SQL is considered too difficult even for the average programmer. We have Object-Relational Mappers to shield programmers from the "complexity" of SQL.
Next candidate: UML. We draw diagrams, pictures that business people can understand or even draw themselves. We have a code generator that translates the pictures to programs, and the user is in control again. This approach, however recent, is already out of favour again. See my earlier post "What's new about Model-Driven Development". The use of DSL's instead of UML does help a little, because a DSL is simpler. But the expectation of some, that the domain specific nature of a DSL means that a domain expert can write programs with it, can safely be regarded as naïve. If only because the domain of a DSL is almost never the domain of the user, but that of the developer.
OK, so new programming languages or tools don't help to bridge the distance between user and programmer. Let's try something completely different. Let's have architects help the business user to describe his business processes, and align the applications to support those. And let's introduce a Service-Oriented Architecture (SOA), so the applications can communicate easily via services. And we need a process engine to directly support the business processes by calling the services in the correct order. "A business user can do this himself."
Sounds great! Only I don't believe a word of it. Firstly, SOA experts keep telling us that a SOA is difficult to implement, and that the potential benefits - easy adaptability of the software to the ever changing requirements of the business - will be only reaped after many years. That is, if the SOA is introduced properly and embedded in the organisation. Governance is the magic word here. So what do you mean: "quick adaptation to changing business requirements"? Secondly, the word governance with me evokes the idea of "programming by committee". And that should be faster than one or two programmers sitting together? Thirdly, now we have architects between the customer and the programmer, making the distance between them greater again. Not to say anything bad about the good ones (I know one or two), but many architects have little affinity with the practical problems ("irrelevant details") facing programmers trying to implement their SOA. I always hear architects complain about programmers not keeping to their guiding principles. Why would that be? And does it sound strange that those organisations that have a separate architecture department are the same whose development process has all but come to an almost complete standstill?
Is there no hope then? Yes, there is. Those projects that adopt a radical agile process, be it eXtreme Programming, Scrum, Evo or whatever, are the same that see a radical improvement in productivity, while actually building what the customer wants. In addition to the fact that the stories sound convincing, I have experienced this myself. That experience was so good that since then I don't want, and maybe even can't, use any other process.
Even though the results are good, there is still much resistance against agile methods. One objection raised sometimes is, that large systems cannot be built without architecture, and that agile methods lead to chaotic and unmanageable systems. The first is true: large systems still need architecture, but the second is not: agile does not necessarily lead to unmanageable chaos. Agile is no substitute for good craftmanship. To create excellent applications in an agile way you need excellent programmers. Only the way architecture is created is completely different. Architecture is a byproduct of software development. It emerges at the same time, not before. Only then you may hope to have an architecture that fits the application like a tailor-made suit.
Applications are so complex nowadays, that they can't be built without architecture. The same complexity is the reason why the architecture can't be thought out completely in advance. If you try you will end up with a rigid system that is hard to change, exactly what that same architecture is trying to avoid. It doesn't have to change, it is thought out in advance. No. Architecture is needed, but can only be developed via an agile process.
Take COBOL, one of the oldest programming languages. COBOL means COmmon Business Oriented Languages. The idea was that COBOL was business oriented enough so that business people could write their own programs with it. This never worked of course. COBOL is widely regarded as a difficult programming language. Good COBOL programmers are hard to find and they are badly needed to maintain the still enormous body of COBOL programs.
SQL: same story. In SQL you tell the computer, or more specifically the database, what you want with the data instead of how to do it. Should be easy enough for business people? Nowadays SQL is considered too difficult even for the average programmer. We have Object-Relational Mappers to shield programmers from the "complexity" of SQL.
Next candidate: UML. We draw diagrams, pictures that business people can understand or even draw themselves. We have a code generator that translates the pictures to programs, and the user is in control again. This approach, however recent, is already out of favour again. See my earlier post "What's new about Model-Driven Development". The use of DSL's instead of UML does help a little, because a DSL is simpler. But the expectation of some, that the domain specific nature of a DSL means that a domain expert can write programs with it, can safely be regarded as naïve. If only because the domain of a DSL is almost never the domain of the user, but that of the developer.
OK, so new programming languages or tools don't help to bridge the distance between user and programmer. Let's try something completely different. Let's have architects help the business user to describe his business processes, and align the applications to support those. And let's introduce a Service-Oriented Architecture (SOA), so the applications can communicate easily via services. And we need a process engine to directly support the business processes by calling the services in the correct order. "A business user can do this himself."
Sounds great! Only I don't believe a word of it. Firstly, SOA experts keep telling us that a SOA is difficult to implement, and that the potential benefits - easy adaptability of the software to the ever changing requirements of the business - will be only reaped after many years. That is, if the SOA is introduced properly and embedded in the organisation. Governance is the magic word here. So what do you mean: "quick adaptation to changing business requirements"? Secondly, the word governance with me evokes the idea of "programming by committee". And that should be faster than one or two programmers sitting together? Thirdly, now we have architects between the customer and the programmer, making the distance between them greater again. Not to say anything bad about the good ones (I know one or two), but many architects have little affinity with the practical problems ("irrelevant details") facing programmers trying to implement their SOA. I always hear architects complain about programmers not keeping to their guiding principles. Why would that be? And does it sound strange that those organisations that have a separate architecture department are the same whose development process has all but come to an almost complete standstill?
Is there no hope then? Yes, there is. Those projects that adopt a radical agile process, be it eXtreme Programming, Scrum, Evo or whatever, are the same that see a radical improvement in productivity, while actually building what the customer wants. In addition to the fact that the stories sound convincing, I have experienced this myself. That experience was so good that since then I don't want, and maybe even can't, use any other process.
Even though the results are good, there is still much resistance against agile methods. One objection raised sometimes is, that large systems cannot be built without architecture, and that agile methods lead to chaotic and unmanageable systems. The first is true: large systems still need architecture, but the second is not: agile does not necessarily lead to unmanageable chaos. Agile is no substitute for good craftmanship. To create excellent applications in an agile way you need excellent programmers. Only the way architecture is created is completely different. Architecture is a byproduct of software development. It emerges at the same time, not before. Only then you may hope to have an architecture that fits the application like a tailor-made suit.
Applications are so complex nowadays, that they can't be built without architecture. The same complexity is the reason why the architecture can't be thought out completely in advance. If you try you will end up with a rigid system that is hard to change, exactly what that same architecture is trying to avoid. It doesn't have to change, it is thought out in advance. No. Architecture is needed, but can only be developed via an agile process.
2009-02-11
What's new about Model-Driven Development?
The introduction of the Model Driven Architecture (MDA) by the Object Management Group (OMG) in 2001 caused a surge of optimism among the relatively few people still interested in code-generation. Finally a standard modelling language (UML) as a basis for code-generation, and consequently the possibility to exchange models between tools of different vendors! Since then the initial enthusiasm for the pure form of MDA has all but died away. Why? Primarily because UML proved less than ideal, in addition to being a modelling language, to serve as a basis for code-generation. Too complicated, too big, not semantically precise enough. And the standardisation proved more theory than practice. Vendors had to change UML in various ways to make code-generation possible, and of course each one did that in a different way. Goodbye interoperability!
But since a year or two the ideas returned in a slightly different and less ambitious guise: now based on Domain-Specific Languages (DSL's) instead of UML. DSL's are quickly getting very popular. Everywhere they are being experimented with, and new conferences about MDD are being organised every now and again.
I can think of at least three reasons for this: (1) DSL's are much smaller and more manageable than UML. (2) The appearance of tools like Microsoft's DSL Tools and openArchitectureWare (oAW) that bring development of DSL's within reach of ordinary developers. Interestingly, these tools are based on one or the other incarnation of the Meta Object Facility (MOF), the core of MDA. This meta-meta model standardises the development of meta models, the abstract forms of languages like UML and DSL's. (3) DSL's fill an urgent need. The complexity of the average JEE application is increasing so fast that few developers manage to keep up. At the back end of an application - the end nearest to the database - the problem is not yet so bad: Hibernate seems there to stay, whether or not as JPA-implementation. The Spring framework stands like a rock, even though it's growing faster than most of us can keep up. But at the front-end - the end nearest to the user - there is no clear winner among the many frameworks. The times that everyone used Struts are long gone. For each new application you have to make a choice between JSF, Wicket, GWT, Spring MVC or Web Flow, or a Rich Client, or many other widely used frameworks. It's simply too much for a regular guy who likes to watch a movie at night once in a while, instead of working through a tutorial (if there is one) of yet another new framework.
So MDD based on DSL's promises a solution for this chaos. You develop a DSL targeted at the domain for the application and a code generator where you can hide all the complexity. Of course for that you still need the rare geek that did manage to keep up with all the latest frameworks, but the rest of us just have to learn the simple DSL. And the geek can pour all his knowledge in the generator, so everyone is happy. So you would think.
Of course we have seen before how new developments promised a lot, but disappointed in the end. Object-orientation, aspect-orientation, agile development, test-driven development, whatever-driven development. So with MDD it will go likewise, probably. But most of these development have still brought some progress, only less revolutionary than we hoped. So the question is, what real progress will MDD bring?
Developing a DSL is hard. Not just the language, but the tools needed to work with it comfortably. So it would be nice if we could re-use it. Therefore the application architecture is often used as domain, instead of the domain of the user of the application. This was done with Mod4j for example. But in doing so the vision of the MDA - separation of business logic from platform specific implementation details - is only partly realised. We have to specify the application in terms of the application architecture, instead of concepts meaningful to the end user. We do make models, but since the appearance of tools like xText, we can develop textual concrete syntaxes as well as graphical ones. So what's the difference between a model and a program. Not much actually. Anneke Kleppe introduced the term mogram in her new book "Software Language Engineering: Creating Domain-Specific Languages Using Metamodels" to emphasise the practical equivalence of a model and a program in a DSL.
So what is new actually? Just this: that the developer works on a higher level of abstraction. For example, Mod4j generates nearly 25 lines of Java or XML code per line of DSL code. According to Frederick Brooke in his book "The Mythical Man Month", the productivity of a programmer is constant in terms of number of statements per unit of time. Real productivity data of DSL's in real projects are not yet available, so what the real gain will be is uncertain. But it certainly looks promising!
But since a year or two the ideas returned in a slightly different and less ambitious guise: now based on Domain-Specific Languages (DSL's) instead of UML. DSL's are quickly getting very popular. Everywhere they are being experimented with, and new conferences about MDD are being organised every now and again.
I can think of at least three reasons for this: (1) DSL's are much smaller and more manageable than UML. (2) The appearance of tools like Microsoft's DSL Tools and openArchitectureWare (oAW) that bring development of DSL's within reach of ordinary developers. Interestingly, these tools are based on one or the other incarnation of the Meta Object Facility (MOF), the core of MDA. This meta-meta model standardises the development of meta models, the abstract forms of languages like UML and DSL's. (3) DSL's fill an urgent need. The complexity of the average JEE application is increasing so fast that few developers manage to keep up. At the back end of an application - the end nearest to the database - the problem is not yet so bad: Hibernate seems there to stay, whether or not as JPA-implementation. The Spring framework stands like a rock, even though it's growing faster than most of us can keep up. But at the front-end - the end nearest to the user - there is no clear winner among the many frameworks. The times that everyone used Struts are long gone. For each new application you have to make a choice between JSF, Wicket, GWT, Spring MVC or Web Flow, or a Rich Client, or many other widely used frameworks. It's simply too much for a regular guy who likes to watch a movie at night once in a while, instead of working through a tutorial (if there is one) of yet another new framework.
So MDD based on DSL's promises a solution for this chaos. You develop a DSL targeted at the domain for the application and a code generator where you can hide all the complexity. Of course for that you still need the rare geek that did manage to keep up with all the latest frameworks, but the rest of us just have to learn the simple DSL. And the geek can pour all his knowledge in the generator, so everyone is happy. So you would think.
Of course we have seen before how new developments promised a lot, but disappointed in the end. Object-orientation, aspect-orientation, agile development, test-driven development, whatever-driven development. So with MDD it will go likewise, probably. But most of these development have still brought some progress, only less revolutionary than we hoped. So the question is, what real progress will MDD bring?
Developing a DSL is hard. Not just the language, but the tools needed to work with it comfortably. So it would be nice if we could re-use it. Therefore the application architecture is often used as domain, instead of the domain of the user of the application. This was done with Mod4j for example. But in doing so the vision of the MDA - separation of business logic from platform specific implementation details - is only partly realised. We have to specify the application in terms of the application architecture, instead of concepts meaningful to the end user. We do make models, but since the appearance of tools like xText, we can develop textual concrete syntaxes as well as graphical ones. So what's the difference between a model and a program. Not much actually. Anneke Kleppe introduced the term mogram in her new book "Software Language Engineering: Creating Domain-Specific Languages Using Metamodels" to emphasise the practical equivalence of a model and a program in a DSL.
So what is new actually? Just this: that the developer works on a higher level of abstraction. For example, Mod4j generates nearly 25 lines of Java or XML code per line of DSL code. According to Frederick Brooke in his book "The Mythical Man Month", the productivity of a programmer is constant in terms of number of statements per unit of time. Real productivity data of DSL's in real projects are not yet available, so what the real gain will be is uncertain. But it certainly looks promising!
Subscribe to:
Posts (Atom)