Blog root post
previous post in category
The traditional form of software development where 3GL code is manually produced is known as elaboration. Effectively the developer directly adds intellectual content at each stage of analysis, design, and implementation by elaborating on what the previous stage produced. That elaborative progression was originally enshrined in the Waterfall model. While the Waterfall has been largely supplanted today by incremental/iterative development models, that is mostly a matter of degree. The basic progression from analysis to design to implementation survives, each is built on the previous, and the developer adds intellectual value at every step -- regardless of the size of the steps.
Originally we developed programs directly in hardware (plug boards) or in Assembly language. Then in the early '60s there was a major paradigm shift in the way software was developed as 3GLs were introduced, initially FORTRAN and COBOL. That paradigm shift represented a major increase in the level of automation in the software industry. Essentially all the mundane details of specific hardware instructions, specific memory addressing, and whatnot were completely abstracted away from the developer's view while compilers, linkers, and loaders did all the dirty work. Thus a large block of developer concerns simply disappeared in the implementation stage.
Currently the software industry is on the brink of another paradigm shift as automation of the computing space moves beyond 3GL compilers. Essentially we are moving to the era of 4GLs where one translates a 4GL solution directly into an executable. It is now possible to describe the solution to an arbitrary problem at a level of abstraction that is at least as far above 3GL programs as 3GL programs were above Assembly programs. This has already been done is specialized niches within the software industry. For example, the RAD IDEs employed for CRUD/USER* processing in applications, that are effectively pipelines between an RDB and a UI, represent essentially a 4GL environment where one thinks of the problem solution in terms of schemas and windows.
The enabling technologies for this automation in general purpose computing are UML and MDA. The addition of an action semantic meta model to v1.5 of UML allows action languages to describe dynamics that have the same level of abstraction as an OOA model. That combination represents a true 4GL. MDA introduces a formalization of transformation that is crucial to being able to implement abstract PIMs against a wide variety of implementation technologies. MDA also allows the formal separation of customer space and computing space concerns represented PIM and PSM.
In separating those concerns, the PIM represents a resolution of only functional requirements that is independent of the computing environment where the software is ultimately implemented. The nonfunctional requirements (size, performance, reliability, etc.) are all addressed by the transformation process. That is, the optimization necessary to resolve nonfunctional requirements in a particular computing environment is completely automated. Thus one skips intermediate PSMs corresponding to OOD and goes directly to the 3GL PSM without any direct developer added value. (One could go directly to Assembly or even machine code but 3GL compilers already do an excellent job on instruction level optimization so there is no point in reinventing that wheel.)
When the transformation engine bypasses the traditional OOD step in development and fully automates the resolution of nonfunctional requirements (i.e., generates 100% of the 3GL code), the process is known as translation. (When MDA is applied to round-trip tools that process individual PSMs and the developer provides the intellectual content, we still have elaboration, even though MDA allows seamless integration of the tools the developer uses.) Translation represents the major paradigm shift that we face today. While the notations and construction techniques remain largely the same and the transformation technique is fundamentally the same as described in the overview post, the crucial difference lies in automation of the computing space -- exactly as it was when 3GLs were introduced. That results in some interesting advantages for translation:
Analysis reuse. The same PIM may be implemented in quite different computing environments with out modification and with complete confidence that the application still logically functions the way the user expects. Thus every application is highly portable.
Design reuse. A transformation engine tailored to a particular computing environment will optimally implement any PIM that it eats there. Thus one obtains full reuse of all of the optimizations for nonfunctional requirements across applications. This is because the resolution of nonfunctional requirements is necessarily dependent upon the particular computing environment and that is all automated.
Productivity. Because of the reuse above, the developer does not have to reinvent the wheel for every application. In fact, an application developer isn't even concerned with inventing the wheel once. In practice this productivity boost is somewhat balanced by the need to provide quite rigorous PIM models; typically much more rigorous that the OOA models in most elaboration shops. The big productivity boost, though, comes through the use of a 4GL. Once gets the same sort of advantages over 3GLs that the 3GLs had over Assembly. That boost is especially apparent when doing maintenance. Identifying changes in a couple of dozen diagrams is orders of magnitude easier than doing so in an MLOC of code. Where I worked before retiring we did an extensive evaluation of the technology before committing to it. (Among other things we completely re-developed an existing 1 MLOC application.) We measured a 10X reduction in maintenance effort -- it was so astonishing we had difficulty getting people to believe our data. But even the Marketeers were astounded by how fast we turned around major changes.
Reliability. The most surprising result was a 50% across-the-board reduction in defects because we had no expectations of any improvement. I can't point to specific practices to make that happen, but I attribute it to the 4GL level of abstraction and separation of computing space concerns. One gets to focus on what is really important to solving the customer's problem and that tends to reduce the opportunities for inserting defects. When fewer defects are inserted, there will be fewer escapes from testing.
An obvious question is: How can one resolve all nonfunctional requirements automatically? Note that the computing space is highly deterministic. That's because everything we do in software must eventually be expressed unambiguously in the simplistic context of Turing and von Neumann computational models. That means everything we express in the computing space is defined with mathematical precision. Thus the entire computing space is amenable to at least some degree of automation. On the other hand, it is no small task to provide that automation because the computing space is vast and very complex. The optimization problems a transformation engine faces are orders of magnitude more complex than those facing the designer of a 3GL optimizing compiler. So it should be no surprise that it has taken almost half a century since the first 3GLs for the engineering problems to be resolved for general purpose computing.
Fortunately the learning curve completed with the turn of the millennium and we are poised on the brink of a major shift in the way software is developed. During a keynote address at a UML World conference a couple of years ago Ivar Jacobson made the prediction that in another decade writing 3GL code would be as rare as writing Assembly is today. I am not sure it will be quite that fast, but it is certainly inevitable now that the technologies have been validated and it only took a decade for FORTRAN and COBOL to almost completely displace Assembly coding.
* CRUD = Create, Retrieve, Update, and Delete; USER = Update, Sort, Extract, and Report.