Home » 2013
Yearly Archives: 2013
In general enterprise software consists of large numbers of systems with complex cross-dependencies and with a high level of heterogeneity and redundancies. The ability of these software systems to address the needs of the enterprise changes over time for the worse. In order to understand the forces at work I will first consider entropy at the system level and then at the enterprise level.
Initially after a software system is delivered, making changes to the system is usually relatively easy. The longer the system is in maintenance and the more changes are incorporated into the code base; the more the ability to make changes to the software system deteriorates  . There are various reasons for the deterioration of software systems. Some of the reasons are due to technology, but mostly the reasons are organizational of nature .
From a technology perspective various best practices exist to counter entropy of software systems. For legacy technologies roadmaps exist to incorporate the latest systems design thinking to address short comings of earlier technologies. As an example IBM provides various strategies to improve the maintainability of RPG programs  .
At a system level the single most significant contributor to the decay of the code base is the lack of routine refactoring to ensure the long term maintainability of the system . These refactoring exercises can for instance be geared towards ensuring the soundness of the design in the face of changing business needs or taking advantage of technology improvements. The lack of timely refactoring may be due to factors such as lack of skilled staff or time-to-market demands.
As such technology is not entirely to blame for the entropy of software systems. Rather, organizational level challenges are the most important contributors to the decay of software systems. Therefore, in order to address software system entropy, change firstly needs to be effected at the organizational level and subsequently at the technology level if appropriate. For example, replacing the existing RPG programs with the latest technology will inevitably in time propagate entropy in the new technology as well if the organizational challenges are not addressed.
Further factors that contribute to the decay of software systems originate at an enterprise level.
Enterprise Software Entropy
The level of entropy found at the enterprise level is significantly higher than that of a single isolated software system. Typically the enterprise architecture is an accidental architecture. That is the enterprise architecture came about organically due to the independent evolution of the various systems and their cross dependencies rather than being planned. Since the various systems have been designed and developed in isolation, redundancies between systems are rife. Many enterprise architectures has grown over many years which results in a plethora of programming languages – often based on disparate programming paradigms – used across systems. With such a lengthy history these systems are frequently hosted on different kinds of servers running a variety of operating systems.
A characteristic of an accidental architecture is the occurrence of stovepipes (or silos). Stovepipes are systems that have been developed to fulfill a particular task without regard for the context of the task within the larger enterprise business process . Since these systems have not been designed to interoperate , this approach sabotages efficiency at an enterprise level . The most prevalent approach to alleviating the effect of silos in the enterprise is through different manifestations of point-to-point integrations. This results in the intertwined enterprise architecture as seen in Figure 1.
The effects of stovepipes are usually acutely noticeable between business units while the intra business unit level effects are often largely hidden from the enterprise. Understandably this causes business and management to have a vague idea of the true complexity of integration projects. This contributes to integration projects being invariable hugely under estimated in terms of cost and timelines . Given this background it is no wonder that an estimated 70% of Enterprise Application Integration (EAI) projects fail . The problems around integration are further exacerbated when e-Business and its associated need for business to business communications is considered.
If we want to address the problems of enterprise software effectively, we have to understand the forces contributing to entropy. These factors can be categorized into day-to-day operations and specific events .
The daily maintenance processes are geared towards the continuous delivery of economic value in the face of ever changing business needs. Enterprise systems generate economic value through value networks. These value networks frequently span multiple applications  since most applications or off-the-shelf solutions are not solving cross-functional or inter-enterprise needs   . In order to support the value network these applications need to be integrated  . Day-to-day operations are typically project and short term focussed instead of being enterprise and long term focussed. This explains the proliferation of point-to-point integrations found in many of today’s enterprises.
Besides day-to-day operations specific events contribute to the decay of enterprise systems . These events include but are not limited to the following:
- The maturation of organizations causes new specialized groups to be included in the Software Development Life Cycle (SDLC) .
- End-of-life events of a supported product  may cause skills to be available only at a premium.
- An enterprise may expand into new territories as a result of off shoring or globalization .
- Mergers or acquisitions can introduce new systems and technologies into the organization. This is likely to result in redundancies at various levels   .
- New lines of business may be introduced to take advantage of new business opportunities . Often business units have their own IT departments with opposing agendas causing infighting between business units .
- Obsoleteness or limitations of legacy technologies may drive technology modernization exercises . Legacy applications further provide challenges in managing a retiring workforce and ever decreasing skills base .
- Government legislation may change the way business is conducted .
It is significant to again note that the role technology play in driving entropy at the enterprise level is negligible. Thus in order to find solutions to enterprise software entropy we cannot consider technical change alone – change must be effected at the organizational level as well.
The traditional solution to the decay of software is to start from a clean slate and rewrite the application using a new technology. Often technology is blamed for the ills of the software industry, but technology is merely a tool at the disposal of the enterprise. The enterprise chose to use the technology because it has a business value. When a newer technology comes along it brings new opportunities for increased efficiencies – it does not nullify the value of the older technology. Therefore a rewrite should not be considered merely on technical grounds (for example the availability of a new technology), but should have a clear business case.
Continuous refactoring can be used to modify an application to take advantage of the latest design principles and stay relevant in the face of changing business needs. However, the prevalence of short term planning in organizations causes the continuous refactoring step often to be skipped altogether. Omitting the refactoring step in the long term leads to code that becomes too costly to refactor, hence the need for a rewrite.
|||D. Krafzig, K. Banke and D. Slama, “Enterprise SOA: Service-Oriented Architecture Best Practices,” Amazon.com, 2004. [Online]. Available: http://www.amazon.com/Enterprise-SOA-Service-Oriented-Architecture-Practices/dp/0131465759/.|
|||M. Juric, P. Sarang and B. Mathew, “Business Process Execution Language for Web Services BPEL and BPEL4WS 2nd Edition,” Amazon.com, 2006. [Online].|
|||L. Patterson, H. Araki, S. Bramley, G. Cobb, J. Eikenhorst, S. Milligan, J. Simons and M. Tregear, “Modernizing and Improving the Maintainability of RPG Applications Using X-Analysis Version 5.6,” http://www.redbooks.ibm.com/, 2006. [Online]. Available: http://www.redbooks.ibm.com/redpapers/pdfs/redp4046.pdf.|
|||D. Cruikshank, “Modernizing Database Access: The Madness Behind the Methods,” IBM, 2006. [Online]. Available: http://www-03.ibm.com/systems/resources/systems_i_software_db2_pdf_Performance_DDS_SQL.pdf.|
|||M. Matsumura, B. Brauel and J. Shah, “SOA Adoption for Dummies,” Software AG, 2009. [Online]. Available: http://zy.xjgame.com/SOBPAO/SOA%20Adoption%20for%20Dummies.pdf.|
|||J. W. Ross, P. Weill and D. C. Robertson, “Enterprise Architecture As Strategy: Creating a Foundation for Business Execution,” Amazon.com, 2006. [Online]. Available: http://www.amazon.com/Enterprise-Architecture-Strategy-Foundation-Execution/dp/1591398398.|
|||R. Schmelzer, “Understanding the Real Costs of Integration,” zapThink, 2002. [Online]. Available: http://www.zapthink.com/2002/10/23/understanding-the-real-costs-of-integration/.|
|||G. Trotta, “Business Process Management (BPM) Best Practices: Dancing Around EAI ‘Bear Traps’,” ebizQ: The Insider’s Guide to Next-Generation BPM, 2003. [Online]. Available: http://www.ebizq.net/topics/int_sbp/features/3463.html.|
|||F. A. Cummins, “Building the Agile Enterprise: With SOA, BPM and MBM,” Amazon.com, 2008. [Online]. Available: http://www.amazon.com/Building-Agile-Enterprise-SOA-Press/dp/0123744458.|
|||F. Kuglin and R. Hood, “Using Technology to Transform the Value Chain,” Amazon.com, 2008. [Online]. Available: http://www.amazon.com/Using-Technology-Transform-Value-Chain/dp/1420047590.|
|||M. Weske, “Business Process Management: Concepts, Languages, Architectures,” Amazon.com, 2012. [Online]. Available: http://www.amazon.com/Business-Process-Management-Languages-Architectures/dp/3642286151.|
|||C. C. Poirier, L. Ferrara, F. Hayden and D. Neal , “The Networked Supply Chain: Applying Breakthrough BPM Technology to Meet Relentless Customer Demands,” Amazon.com, 2003. [Online]. Available: http://www.amazon.com/The-Networked-Supply-Chain-Breakthrough/dp/1932159088.|
The business rules which form the basis of the unique product offerings of corporates are typically embedded in bespoke systems. Gartner, in their 2013 IT Cost Optimization (ITCO) webinar , states that the biggest ITCO opportunity is found in reducing the per function point application development costs. To this end corporates may look towards writing their own frameworks in order to achieve ITCO targets. Although building a framework may be a viable option, it is certainly not a guarantee for success.
Benefits to be Reaped from a Framework
Let us first look at the benefits companies hope to reap from creating their own frameworks:
- Developer productivity – Reuse is an often cited benefit of frameworks  . Instead of different business units creating their own but slightly different business capabilities, business units can leverage off the common business capabilities provided by the framework.
- Shorter time-to-market – With readily available common business capabilities, the assumption is that business units can focus on only developing the delta of differentiating business capabilities unique to their particular business unit. The ability to focus on the differentiating business capabilities of the business unit; enables business units to push new products into the market much faster. This in turn results in prompt return on investment (ROI).
- Reduced maintenance costs – Since common business capabilities are only implemented in one place, it is developed and tested once. When the business rules change, changes to the relevant business capabilities can be made in one place.
Complexities of Designing a Framework
If there are so many benefits to having a framework, why are all corporates not creating one? Frameworks are orders of magnitude more complex to design due to the higher level of reuse, the need for constant evolution and backward compatibility  .
- High level of reuse – In software development reuse is often considered a virtue    . When stable and robust artefacts are reused, this is certainly true. However, reuse can become a vice when the artefact in question is in frequent flux and/or brittle.
- Constant evolution – Requirements are seldom stable. As requirements change, the framework will have to evolve gracefully.
- Backward compatibility – If clients are forced to make major changes to their existing code every time a new version of the framework is released, developer productivity is hampered rather than accelerated. Backward compatibility is not merely achieved by keeping API signatures consistent: It also wreaks havoc in instances where the underlying behaviour of the framework changes between versions.
Impact of Getting it Wrong
Multiple incompatible variants of a framework are a sure sign of a framework failing. Please note: I am not talking of legitimate versions here which forms part of a roadmap. Rather, I am talking of accidental variants of the framework that come into existence due to attempts at addressing flaws in framework. These variants may come about for a number of reasons.
- Use cases of the framework may not be clear resulting in teams inadvertently using the framework incorrectly. That is the framework is accidentally used in a non-backward compatible way.
- Brittleness of the framework may force teams to independently hack the framework into a vaguely working state resulting in multiple incompatible variants of the framework.
- The framework may not have been designed with backward compatibility in mind. The effect is that the effort to upgrade to a later version of the framework becoming an obstacle in itself.
Needless to say when multiple variants of a framework exist, it destroys all intended benefits:
- Maintenance costs will increase due to the inherent complexities associated with frameworks. With each variant of the framework evolving independently, maintenance costs are increased further.
- Developer productivity will be decreased due to increased complexity which will cause an increase in analysis time. These additional efforts will increase the time-to-market which hurts ROI.
Guidelines for Making a Framework work for Bespoke Software
Does this mean frameworks are a bad idea for bespoke software? Certainly not! Here are some guidelines for making a framework work for bespoke software development:
- Design stepwise – Do not attempt to provide the solution to all problems in one go. Particular good advice from both  and  is that a new feature should rather be delayed or not added at all when adding it will cause the integrity of the framework to deteriorate
- Build slowly – You are going to make mistakes, especially if you have a team with limited experience in building frameworks. Remember, once you have added features into the framework, removing them can be extremely difficult to almost impossible. Building the framework slowly gives the team the opportunity to learn without making too many commitments to their users that they cannot honor in the long run.
- Manage business expectation – An appropriate framework has the ability to reduce time-to-market substantially once it has reached a critical mass of functionality. A key point is that while it is in the process of being built, it slows down time-to-market significantly due to the inherent complexities in building a framework. Business expectation has to be managed accordingly.
Frameworks certainly have the ability to decrease the per function point application development costs, but the cost of naively embarking on a framework journey should not be under estimated.
|||K. Potter, “2013 IT Cost Optimization: Strategy, Best Practices and Risks,” 5 December 2012. [Online]. Available: http://my.gartner.com/portal/server.pt?open=512&objID=202&mode=2&PageID=5553&ref=webinar-rss&resId=2232818&srcId=1-2973089105&prm=tpc-itco. [Accessed 15 Febuary 2013].|
|||M. Fayad and D. C. Schmidt, “Object-Oriented Application Frameworks,” vol. 40, no. 10, 1997.|
|||D. C. Schmidt, “Applying Design Patterns and Frameworks to Develop Object-Oriented Communication Software,” in Handbook of Programming Volume I, MacMillan Computer Publishing, 1997.|
|||J. Tulach, Practical API Design: Confessions of a Java Framework Architect, Apress, 2008.|
|||K. Cwalina and B. Abrams, Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries, Addison-Wesley, 2009.|
|||M. Fowler, K. Beck, J. Brant, W. Opdyke and D. Roberts, Refactoring: Improving the Design of Existing Code, Addison Wesley, 1999.|
|||S. McConnell, Code Complete: A Practical Handbook of Software Construction, Microsoft Press, 2004.|
|||R. C. Martin, Clean Code: A Handbook of Agile Software Craftsmanship, Prentice Hall, 2009.|
|||K. Beck, Implementation Patterns, Addison-Wesley, 2008.|
|||J. W. Ross, P. Weill and D. C. Robertson, Enterprise Architecture as Strategy: Creating a Foundation for Business Execution, Harvard Business School Press, 2006.|
|||M. Kanat-Alexander, Code Simplicity, O’Reilly®, 2012.|