John Steensen’s Blog

The Complexity Ceiling May 12, 2009

Software systems  are some of the few human endeavors that seems to defy basic rules about how complex a system can become before it loses the ability to  adapt and survive. If a hundred features are good then a thousand features must be much better. There seems to be almost no ceiling on the complexity that software developers are willing to inject into a software product. Every line of software used to be viewed as both as an asset AND a liability. But with modern QA and regression testing tools it has become relatively cheap to carry legacy features and the code that embodies them. Legacy features, like gene pairs gone bad, can have devastating effects when interacting in unexpected ways with new features.  

Pride Cometh Before a Rise

Every feature has a story. And that can be a problem. When the primary currency paid  for developement effort is attribution then the system surrounding and supporting that effort must have a well-managed and promoted attribution mechanism. For example, in the open source software development community that mechanism is the possibility of rising to the “inner sanctum” of developers that have final approval for which features and implementations will make it into the “core” code line. This group is, of practical neccessity, a relatively small and elite group. So, while there are certainly those individuals who donate their efforts for the “common good”, I suggest most expect some form of attribution from the community they are contributing to. And when this attribution mechanism is effective and well-managed then the number of contributors will increase. In the commercial world monetary compensation takes on part of the role (stock, salary, bonuses) but attribution to the individual and/or team is still very important. So the addition of features keeps on coming.

Aggressive Feature Deprecation

Experience has shown that the apriori feature selection process often results in software features that of limited practical value (not discounting their marketing value) or are implemented in a less than optimal fashion. This “plant 1,000 flowers and let them bloom” approach may have utility in allowing the evaluation of many different features but only works if the ecosystem has a mechanism to get rid of the “ugly” flowers. Although software feature utility could be determined by surveys and polls those polls could lead to “feature support groups” pulling a “50 cent army” on the community. We need a way to systematically organize the deprecation of features with little utility. Note that “little utility” is different than “little used”. For example, a data recovery feature may be (hopefully will be) little used but could have very high utility.

I believe a process I call Aggressive Feature Deprecation or AFD (not to be confused with AFD – A F***ing Disaster) should be applied to all large software systems. AFD would be implmented as a standardized, very-lightweight, mechanism to record the use of each tracked feature into a log file in an industry standard format. This log file, under the complete control of the user or, in the case of commercial software, the license holder, could be viewed before it was sent, either automatically or manually, to the software creation or management entity. By the automatic monitoring on the use of specific features there could be objective and usage-based discussions about each and every features utility. By adding quantitative information to community-based information better decisions could be made about which legacy features should continue to receive support and which should be deprecated and how fast they should be deprecated. And so, if there is a complexity ceiling, we can buy some more headroom before we hit it.