University of Durham
School of Engineering and Computer Science
in Software Maintenance
John R. Foster
The objective is achieved by setting up a general model of maintenance, whose most detailed features are the technical actions which accomplish the task, but which also encompasses management controls and their financial consequences. Parts of the model are used to develop a formal description of the process of selecting change requests by priority and planning the timing of the software release.
The calculations of the model are implemented as a computer program, whose inputs are parameters describing a set of change requests and salient project details. The program is exercised with specimen data, and planning options and their outcomes are explored. Outcomes are expressed in terms of timescales and returns on investment. It is concluded that such analysis is not only desirable, but achievable in actual commercial projects.
I would like firstly to thank my supervisor, Malcolm Munro, for his guidance and encouragement throughout this project. Without his support, I could not have completed the project. I would also like to thank BT for their invaluable support and sponsorship of the work.
The management and staff of the research library at BT Laboratories have been tireless in their enthusiasm to help, and to them I record my deep thanks.
Many individuals have helped in ways that cannot be attributed directly by
references in the text. Some have been my managers at different stages (and all
actively supportive); some have provided their own bibliographies or insights
through discussion; and a noble few provided help with the proof-reading. They
Colin Archibald, Andy Beasor, Keith Bennett, Roger Browne, Frank Calliss, Hilary Calow, Peter Cochrane, Bill Collins, Mel Colter, Simon Cooper, Nigel Coulter, Roy Everett, Nigel Fletton, Bob Frost, Ian Garrett, Bob Higham, Ann Horsey, Charles Jackson, Adrian Jolly, Darius Karkaria, Rachel Kenning, Hans Kiekuth, Ray Lamb, Bridget Leathley, Jan LeFèvre, Steve Lineham, Dave Lumby, Mark Norris, Fay Owston, Tim Pennick, Kearton Rees, Peter Rigby, Andrew Rombach, Sinclair Stockman, Alan Stoddart, Richard Storey, Mike Tilley, Nigel Titley, Bob Wachtel, Richard Warden, Ivan Warner, Paul Whittle, Phil Williams and Neil Winton.
The copyright of this thesis rests with the author. No quotation from it should be published without his prior written consent, and information derived from it should be acknowledged.
The design, development and subsequent maintenance of computer software are the realm of what is generally known as the discipline of software engineering. The accepted use of this term dates from 1968 when an historic conference was organised in Garmisch, Germany, under the auspices of the Study Group on Computer Science of the NATO Science Committee.
In a background note to the conference, it is recorded [Nau76, p. 5] that:
In late 1967 the Study Group recommended the holding of a working conference on Software Engineering. The phrase "software engineering" was deliberately chosen as being provocative, in implying the need for software manufacture to be used on the types of theoretical foundations and practical disciplines, that are traditional in the established branches of engineering.
Twenty-five years on, software retains its special reputation as one of the least disciplined of engineering subjects, despite its all-pervasive nature and the fact that the great majority of software developments are successful and productive. It is often pointed out, and fairly, that software engineering is a very young discipline when compared to others; but we would also stress two factors that have little to do with maturity:
After construction, both hardware and software products enter phases that are termed "maintenance," but here again there are strong differences. The great majority of hardware maintenance consists of the replacement of components that have failed through stress or age; actual design changes are likely to be few and limited in their scope. Hardware maintenance by and large attempts to restore original functionality, with a high probability in any particular case of success. Software maintenance, on the other hand, consists entirely of design changes; and the extreme sensitivity of the product's behaviour to tiny differences in implementation makes such changes intrinsically more likely to fail in unexpected and even catastrophic ways.
We do not, then, expect software engineering to evolve with time into "just" another engineering discipline, with the same attributes and predictability as others. It will continue to improve, but the scope for convergence is restricted.
The thesis came into being as the result of a wish on the one hand, to provide an element of formal description to the software maintenance process and on the other, to show that the formal description could lead to benefit in a commercial setting. The author's past experience as head of a team engaged in the software maintenance of telephone switching systems played no small part in this motivation.
All organisations make investments in the tools and equipment needed for them to function, and for most companies, computer programs make up part of that inventory.
Investments in hardware (machine tools, buildings, vehicles etc.) are costed and accounted for under rules that are well understood. Capital investments are shown in company accounts as asset values to which depreciation applies, and investment decisions are based upon comparisons with the estimated earning power of the equipment involved. Maintenance costs are generally taken into account in these initial calculations, and regarded thereafter as relatively fixed expenses that have already been allowed for. Future design changes are not taken into account unless they can be clearly predicted; rather, they will be treated as though they were fresh investments when the need arises.
Investments in software are, at the development or purchase stage, amenable to similar calculations (although the costs are rarely reflected in company asset sheets). The provision for maintenance, however, meets with serious difficulties. Routine maintenance of the predictable kind is entirely absent, yet it is known from past experience that for the average product, software maintenance will cost as much again as development. This maintenance is the accumulation of many unpredicted design changes. Often the cost may be ignored altogether in the initial planning; if it is not, the alternative is often seen as being to assume some overall average figure, and to enter that into the cost calculations. Either way, when the design changes are later proposed, there is no foundation for their cost-effectiveness to be assessed in the same manner as for hardware.
The thesis proposes and demonstrates a method for the assessment of decisions to undertake software maintenance tasks in response to change requests. The assessment method uses financial criteria, which means that alternative investments may be considered and compared. The method derives from a formal description of the maintenance process, and is shown to be capable of implementation for potential commercial use.
The thesis takes as its basic premise that decisions taken during software maintenance should be informed by financial criteria, so that investment in maintenance can be justified not only in itself but against other potential applications of funds.
In order to meet its aims, the thesis must:
Chapter 2 begins with some general remarks on the software maintenance process and what is known (or believed) about it. A definition of software maintenance is established, and common attitudes towards it are discussed. The current state of knowledge is briefly reviewed, followed by discussion of current research efforts.
Chapter 3 concentrates on models that have been proposed for the maintenance process, showing the variety of approaches that have been adopted.
Chapter 4 gives a qualitative description of the model that has been developed for the work of the thesis. It is divided into seven levels; one (the team level) is central to the thesis, but the existence of the others is important in demonstrating the model's ability to span from technical detail to company asset base.
Chapter 5 expands the model's description, but now concentrating on a quantitative description of the team layer and the parameters associated with it. Every parameter and derived quantity is described in this chapter, which thus comprises the complete formal model on which the thesis is based.
Chapter 6 returns to a more qualitative style and describes how the calculations of the previous chapter may be manipulated in the evaluation of maintenance options.
The calculations are well suited to a computer program and Chapter 7 describes the implementation that has been developed, together with its limitations.
Chapter 8 then brings together the formal model and its implementation, by setting up a number of maintenance scenarios and showing how they are evaluated. The scenarios are based on an exercise carried out on an actual maintenance project, but the details have been somewhat disguised in the interests of commercial confidence.
Finally, Chapter 9 concludes the thesis by summarising the results from the previous chapters in the light of the criteria for success, describing the opportunities for further research, and listing the conclusions of the thesis as a whole.
Different people hold different views on what is meant by the term "software maintenance", and a few even refuse to use the term at all. In this chapter we will begin by examining the main competing definitions of the term, and establishing the definition that will be used during the rest of the thesis. Section 2.3 considers people's attitudes towards maintenance, both historically (including discussion of the reluctance of many to use the term at all) and in the light of more recent and positive changes. Section 2.4 offers some perspectives on maintenance, citing the growth of research and the great range of project sizes. Section 2.5 summarises present knowledge about the overall costs of software maintenance. Section 2.6 then looks at some of the research efforts that have led to proposed and actual improvements in maintenance operations. In Section 2.7 we consider the motivation for research and the importance of finance in it, establishing the motivation behind the work in this thesis.
There are at least three definitions of software maintenance that are in widespread use. Two of them are frequently used in industry, while the third is preferred by researchers and is also the official definition according to the IEEE (The Institute of Electrical and Electronics Engineers, Inc.)
The first and perhaps most restrictive view is that maintenance is the activity of correcting errors in operational software [Fro85]. It is used in some areas of data processing, where separate teams of people work on the same programs: one team will be adding functionality to a base release, while another team will be responding to user problem reports and building new versions of the programs on a regular basis. The advantage is that each team is geared to its particular timescales; a disadvantage is that changes must be implemented twice, if the later enhanced release is to reflect user requests already granted.
The second view is also found in data processing, and is related to the first. Instead of distinguishing maintenance activities by the reason for change to the programs, it classifies as maintenance any change that is expected to take less than a certain amount of effort. The actual threshold varies between organisations, but typical values range from a few days to (less commonly) a few weeks or even a year [FH79]. In a variant of this definition, it may also be decreed that all corrective actions are classified automatically as maintenance -- a combination of the two views.
The two views may be described respectively as the technical and the economic approaches. What they have in common is that each provides a way of avoiding the calculation of detailed costs and benefits for individual changes: instead, overall levels of activity are restrained by fixed budgets, and the goal is to achieve as much as possible within those budgets.
Under either view, it is apparent that user requests which are classified as maintenance are acted upon more quickly than those that are classified as development. Users are not slow to react to this and may well try to turn the distinction to their advantage, for example by disguising enhancement requests as requests for the correction of errors [Swa79, Gre84]. A further corollary is in planning the activities: maintenance requests are typically given much less scrutiny than enhancement requests before they are authorised. This makes them even more attractive to users as a means of getting changes made, but it also reduces the emphasis on seeking cost-effectiveness in maintenance.
These reasons are enough to make the commercial definitions unattractive from the point of view of research study, and this is given added force by the fact that the process of making any change to operational software is for the most part independent of the reason for that change. Tools and techniques that benefit error correction also benefit the making of enhancements, and the risks inherent in each activity are the same. We will therefore adopt a comprehensive definition: that software maintenance is the set of activities involved in making any change to operational software, for whatever reason. This is consistent with the usual interpretation of the standard IEEE definition [IEE93]:
Software maintenance is the modification of a software product after delivery to correct faults, to improve performance or other attributes, or to adapt the product to a modified environment.
Although people's attitudes towards software maintenance are changing, it has long been regarded as a second-class subject and has only comparatively recently been seen as a respectable subject for the attention of researchers. The widespread "quick-fix" view must take much of the blame for this; coupled with the lack of attention to cost-effectiveness, it leads quickly to a view among management that software maintenance is an expensive chore rather than a productive and profitable activity.
This misperception has had several side effects. In their pioneering survey of software maintenance [LS80], Lientz and Swanson reported that senior managers may add to the problems of maintenance if they resist involvement in it through their perception of the effect on their own recognition and career advancement. That such attitudes exist is supported by other surveys as well as by direct observation; for example, a survey in 1986 explored whether managers' attitudes towards maintenance were favourable, neutral or unfavourable. Only 17% of the 158 respondents viewed it favourably, with 38% being neutral and the other 45% taking an unfavourable attitude [Cha86]. One can only speculate on the effects of this on their maintenance staff.
One result of negative attitudes has been the renaming of activities that we would classify as software maintenance. A selection culled partly from the literature but mostly through conversations between the author and maintainers and their managers is as follows: further development; production programming [Can72]; current engineering; Phase 2; post deployment software support (this one from the US military); software continuation engineering (advocated in [Jon81] but not seen in practice by this author); post-release development; post-design services; phased development; installed system development; software sustaining engineering. None of these terms has seen widespread use, perhaps because of a feeling that if any did it would attract the same reputation and lead to a further search for synonyms.
If the above summarises the traditional attitudes towards the subject, one can at least point to a growing positive interest as the business and economic importance of software maintenance gains recognition. From a negligible base, we shall see in the next section the sharp rise in the volume of published results over the last decade or so.
The origins of the growing recognition of software maintenance may be traced back to the decade between 1970 and 1980, in which numerous papers drew attention to the fact that maintenance is a greater consumer of resource than development. Estimates of the actual proportion of software expenditure devoted to maintenance varied between about 40% and 75%, with general agreement that whatever the actual percentage, it was rising steadily and could be expected to continue to do so. Figure 2.1 illustrates the published estimates: each point shows the date of publication, and the estimated percentage of software expenditure devoted to maintenance.
Not all maintenance projects are created equal, and differences in scale and in application can demand substantial differences in approach, from the one-person "team" to the dedicated department and from the casual to the highly formal.
To illustrate differences in project size, it is customary to refer to the number of lines of code that the project contains. The line count is an approximate measure in that many different definitions are possible, but here we are interested in gross differences only and will tolerate that approximation and more. It has another disadvantage, however, in that it does not encourage visualisation of the quantity described. The difference between a 100,000 line project and a 1,000,000 line project is considerable, but to most people the numbers convey little real impression of the magnitude.
In response to this latter problem, this author introduced at a conference presentation in 1987 [FM87] an alternative expression of the line count metric, which has become known as Foster's metric. Its formula is simple if anachronistic: it is the length of line printer paper required in order to print the entire program. The basic unit is the mile in countries that use that measure of distance, or the kilometre otherwise. Line spacing at the line printer standard of 6 lines per inch is assumed, which with a little allowance for page margins gives conversion factors of 400,000 lines per mile, or 250,000 per kilometre.
On this basis, one tenth of a mile represents a baseline where programs are of more or less human dimensions. In 1980, published survey results indicated that the average application program in data processing was about 23,000 lines of code, or 0.06 mile, and the same survey indicated that the average member of maintenance staff was responsible for about 38,000 lines (0.1 mile). From informal observations by the author, about twice this baseline would be the extreme limit of what one person could handle alone, and such a person would need to start with a high degree of knowledge of the system being maintained. Systems of about this baseline size were classified by Capers Jones as medium-sized systems [Jon77]. These system sizes and those still to be described are shown graphically in Figure 2.3.
Systems larger than this are correspondingly more difficult to maintain, both because of the sheer volume of code and because their applications tend to be somewhat exotic. If we look around the 1 mile figure, a typical example would be the on-board flight control system for the Space Shuttle (actually about 500,000 lines, or 1.2 miles). Capers Jones classifies this as a "large" system, and the fact that it is also in the life-critical class makes it essential to adopt a very formal maintenance strategy with strong emphasis on process quality and improvement. That its maintainers have risen to the challenge is evidenced by the fact that the shuttle has never encountered a serious error in its flight control software, and by the estimate [Kel92] that this is the world's only maintenance project to have achieved the top level of 5 on the ratings scale of the SEI Model. (It can only be an estimate since the formal SEI Model does not directly address maintenance).
Between one and five miles is the range in which most telephone switching systems are found. Projects of this size are classified by Capers Jones as "superlarge" -- his highest category. These systems are not regarded as life-critical in the same sense as the Space Shuttle, but their reliability requirements are typically for not more than one major failure from any cause per 40 years. Again, it is not the size alone that makes them interesting as objects of maintenance.
If there is a practical limit to software size, it may at present lie at and above about 20 miles (8,000,000 lines). Inventories of programs can and do exceed this figure, but these are aggregations and not single systems. The possibility of achievement of this sort of size was publicly debated in 1983, when the now-cancelled Strategic Defense Initiative project was first mooted. The original estimate was for 10,000,000 lines (25 miles) of control software [Fle83].
Figure 2.1 showed the wide range of estimates of how much of software expenditure goes on maintenance, and until recently the accepted consensus has been one of high and rising costs. As an absolute amount of money, maintenance costs are indeed high; but the evidence supports neither the assumption that it accounts for the great majority of software costs, nor the assumption that the proportion is rising over time.
In a paper by the author in 1991, these assumptions were examined and rejected, at least for data processing applications [Fos91]. The paper argues that maintenance represents a constant percentage of software costs, at between 50 and 55%, and suggests that managers achieve the constancy by controlling the rate at which old programs are phased out and replaced by newly developed ones. An implication of this is that cost-effectiveness may not yet be a widely applied factor in such decisions.
The paper goes on to suggest that, if program lifetime is indeed being controlled to keep maintenance costs a constant proportion of expenditure, then overall improvements in the maintenance process will be measured by extensions in average program life and not by changes in overall expenditure. This is unfortunate, because program lifetime has been little studied [Ken90]. The general surveys that have been carried out have generally been patterned on [LS80], which contained no representative questions on the age of the software portfolio. (Program age did figure in part 2 of their questionnaire, but only to record the age of a sample application which was itself not randomly chosen). However, more recent work in Japan shows that the issue is now beginning to be addressed [TT92].
There have been numerous attempts over the years to "tackle the maintenance problem" with a variety of weapons, which can be classified broadly into two main types: improvements to the development process in the hope that better designed programs will require less maintenance; and direct attempts to improve the maintenance process itself. Early developments in high level languages and in the adoption of the structured programming discipline are these days taken for granted as better practices than their predecessors, and there have been many attempts to capitalise further by expanding the ideas.
So-called fourth generation languages (4GLs) have been one such attempt. The name covers a variety of development tools (being more a marketing than a scientific term), but it is generally applied to application generators that help designers to solve problems within more or less restricted domains. The expectation was that by promoting extremely rapid development times, maintenance would all but cease to exist, being replaced by further redevelopment as soon as change was required. That initial promise is now seen as somewhat flawed, though the reasons are not purely technical. 4GLs are by their nature designed for fairly specific application domains, which means that a wide variety of them have been produced. The process of standardisation, so much a feature of the more successful third generation languages, was notable by its absence, and with reduced market share for each offering the tendency was for each 4GL to remain the property of its sole manufacturer. Users who were perfectly happy with the performance of their chosen 4GLs now find themselves vulnerable to their suppliers being taken over or otherwise going out of business; and if that does not happen, they then find themselves locked into using whatever range of machines their 4GL supplier decides to support. Not a few former 4GL converts are now in the process of migrating their software back to COBOL (e.g. [McD92]).
Formal methods have long been regarded as a great future hope for software, with their promise of rigorous mathematical proofs of correctness of programs. Claims have been made that programs that have been proven correct will require no maintenance; but these claims fail to take into account that corrective maintenance accounts anyway for only a minority of the effort. Formal methods have not yet advanced to the stage where they can be used on large-scale projects, and there has been little study of how incremental change will later be imposed on the mathematical proof structures.
Particularly in recent years, object-oriented programming has achieved considerable acceptance in software development, and the flexibility of its attributes such as inheritance and dynamic binding have been promoted as being automatically better for maintainers too. These claims have been regarded by many as self-evident; yet studies have called the benefits into question, since by their nature they can put program analysis and comprehension beyond the range of present-day tools and methods [WH91, LMR91].
Of direct assaults on the maintenance process, great efforts (and progress) have been made in the areas of reverse engineering and re-engineering [CC90]. Reverse engineering is the first stage of re-engineering, and it refers to the process of analysis of the source code in order to (re)discover underlying structures and promote understanding of the operation of the existing code. Gaining such understanding has been reported to take large amounts of programmer time: a survey of 800 programmers by the Mellon Bank found that they spent 29% of their time simply studying the documentation and code [Til87]. Other such estimates have been 50-90% of maintenance time [Sta84], and 30-60% [DBSB91]. The broader process of re-engineering starts with code analysis, but has the further goal of using the discovered structure as a base for reconstruction of the program, such that the structure is more cleanly reflected for the benefit of subsequent maintenance operations. The state of the art in both these subjects is well reflected in [vZ93].
Running through many of the published papers on software maintenance is a familiar litany. It runs something like: "Software maintenance is little understood, yet large amounts of money are spent upon it. Any new understanding must be good, and within such large expenditure even small technical advances must be worth many times their cost."
This is admittedly something of a caricature, yet a maintenance manager contemplating investment in a new maintenance tool, or in training costs for a new method, is likely to have little more available by way of justification. The problem here is that justifying expenditure on software within the normal business is itself a little understood process. It is conventional to appraise investments according to their costs and expected future income, but the future income of a program which supports a business without directly generating revenue is difficult to calculate and currently the subject of research [Cle91, WDK93].
For managers in other fields, the ideal way to justify expenditure is to put forward an assessment of the financial implications of a proposed change in an activity. The calculation will include both costs and benefits, and will compare the net benefit with the costs involved. The result will be expressed in terms of a return on investment, which will also enable competing demands on available money to be ranked against each other. (Should we buy this new maintenance tool, or should we spend the money on additional advertising instead?) We suggest that software maintenance is more amenable to this kind of calculation than is software development, and the model put forward in the thesis will make its calculations on that basis.
Software maintenance is the process of making incremental changes to a software product in order to retain and enhance its value in the light of changing demand. Historically, it has suffered from negative attitudes towards it. However, recent years have seen changes in those attitudes and not the least of the effects has been a rapid rise in the number of research publications related to it. Maintenance accounts for just over 50% of all software expenditure, which realisation has no doubt acted as a stimulus in that direction.
The difficulties of maintenance are not the same for all projects, as can be seen from a comparison of project sizes. The great majority of programs are relatively small, but extremely large ones do exist and these tend to have critical reliability requirements by the nature of their application.
The expense of software maintenance in an organisation can be considerable, yet maintenance actions are not subjected to the same investment analysis as decisions in other areas (including software development). Maintenance projects are typically controlled via fixed budgets, and the calculation of return on investment does not take place.
This chapter examines various models that have been put forward to explain and support the maintenance process. The classification takes an evolutionary view, beginning with the earliest models and progressing to the more recent (and broader) offerings.
In Section 3.2 we review a type of model that has been put forward from the early 1970s. It concentrates on the so-called modification cycle, which represents the sequence of actions taken by maintainers as they work through from change request to new release. Many variations exist, but at heart the models are prescriptive sequences of operations to be performed.
These models largely held sway until the mid-80s, when an expansion of viewpoints is apparent. The modification cycle is still central: but now there is new emphasis on other entities in the process such as types of knowledge and forms of documentation appropriate at each stage. Section 3.3 considers these developments.
The above models all concentrate on maintenance as practised by maintainers. The next set to be presented broadens the perspective by considering maintenance as a continuously managed process, thus introducing the management aspects. The modification cycle is still understood to comprise the basic set of activities being managed, but the new generation of continuous process models looks for common elements across longer time periods, with emphasis on measurements of activity and effectiveness. These models are discussed in Section 3.4, and include models which take into account externally imposed constraints such as the SEI model of software maturity.
Along with the evolution of models has come a gradually increasing awareness of the importance of costs and benefit measurement, alongside the technical aspects of maintenance. Section 3.5 considers possible future developments along that path, and some of the obstacles to their creation.
These models present the maintenance process from the viewpoint of the individual maintenance programmer. They describe the sequence of actions taken in response to a change request, the sequence being known as the modification cycle [EM82]. The main models of this type up to 1983 are summarised in [CB86].
An early example is given in [Liu76]. Just three steps are enumerated: understanding the problem; designing new program logic; and incorporating the revised logic into the program. Testing is treated as an activity external to the model, though of no less importance for that.
Later models extend the modification cycle to include the testing phase, and expand to varying degrees the number of stages. Many also include at the start a stage of problem analysis: thus in [Sha77] we have: problem verification; problem diagnosis; reprogramming (including rebuild); and baseline verification and validation. Similarly in [McC81] the four stages are: program understanding; identification of objective and approach for the modification (including detailed design); implementation; and revalidation. The second and the last of these stages are then broken down into respectively four and five sub-stages.
These and other modification cycle models differ in detail and in exactly where they consider the modification cycle to start and stop, yet their similarities are stronger than their differences. All have in common their view of maintenance as a sequence of stages, with only incidental reference to the objects being maintained and the information required for the process.
The summary of modification cycle models referred to above [CB86] was intended as more than a simple review. It used the commonalities between the models to derive a complementary list of the items of information required by the maintainers. These included information on the original requirements and specification, the architectural and low-level design and so on. These, along with the source code, are the entities manipulated by the maintenance process.
Later models take this idea and expand on it, and in [HQ92] there appears a process description which starts with a modification cycle but goes on to define actors and their tasks, and documents and their information flows. Actors include review boards as well as maintenance team members; documents are such as the software problem report and software change proposal.
There is even more expansion in [CC92]. This paper, which along with [CB86] has Collofello as a co-author, presents maintenance as a sequence of tasks (stages), which manipulate entities (documents) in accordance with pieces of knowledge (of the information in documents; of other information for which there may be no formal record; and of how to perform tasks). Their model contains 17 tasks, 13 document entities and 29 knowledge items.
The IEEE Computer Society have for some years been studying the various models available, with a view to issuing a standard in the area. That has now been done, and IEEE Std 1219-1993 contains a seven-stage modification cycle augmented with the relevant document entities. The description of each stage includes the list of documents required, the detailed processing steps, the control actions (effectively a checklist to ensure correct processing), and the list of documents modified and generated. The stages begin with examination of the change request, and end with delivery and installation.
The models so far described concentrate on how to perform maintenance. A more recent type of model takes a wider and more abstract view, seeking to identify the opportunities for improvement in the way the process is carried out.
Boehm's Spiral Model [Boe86, Boe88] is an early example of this abstraction process. It emphasises the setting of objectives and the evaluation of risks, and generally views maintenance more from the viewpoint of the manager than the maintainer.
Process improvement implies process measurement, and one such measurement system is described in [RU89]. The paper gives a model of a NASA maintenance process, and describes how a goal oriented approach was used to define a measurement programme, whose aim in turn was to identify parts of the process that were candidates for improvement.
The general theme of process quality came into prominence with the work of the Software Engineering Institute (SEI) in establishing guidelines for the evaluation of quality in software suppliers [Hum89]. The result of this work is the SEI Model, which defines five levels of process quality and contains questionnaires for carrying out the assessment process itself.
Despite its influence generally, the SEI model directly addresses only software development. This is, from the maintainer's point of view, a deficiency to be rectified. Work at the University of Durham in the UK has led to the development of a specific model for maintenance process improvement, in which the steps of analysis, design of measurement, and modification of the process are described [HB92]. Simultaneously, the SEI Model itself is being reviewed by some of its users, who are proposing extensions for the assessment of maintenance operations [Dre92].
Most maintenance managers would suggest that the costs of the software maintenance process are only too apparent, and to their managers in turn a primary objective is likely to be cost reduction.
Modification cycle models and entity models lend themselves to cost reduction exercises, because in breaking down the maintenance process into smaller stages they allow cost measurements to reveal the most expensive or resource-hungry tasks. These then become candidates for closer inspection.
The drawback of this approach, however, is precisely in its focus on costs alone. There is clear advantage in streamlining a task if the delivered quality of the product can be preserved, but it is impossible to identify those process changes which might actually increase maintenance costs while still delivering a higher net value in the end.
The previously cited method in [CC92] is for the most part cost-based in this manner, but it introduces a benefit-based element in suggesting that a causal analysis of errors could reveal those tasks whose improvement might yield a high return on investment. The suggestion is not expanded upon, and there is no indication as to how the return on investment might be calculated.
In a true cost/benefit model as we would define it, the net benefits of maintenance operations would be measured (or predicted) as the effects on the organisation's profits as a result of the changes and expressed in monetary units or more generally as a return on investment. One such model is the subject of this thesis.
Early models of the software maintenance process concentrated on the sequence of steps followed by the maintenance programmer. Model development has since expanded on this view, though many still retain the sequence as a component.
Process improvement models seek to evaluate the overall quality of the process and thereby identify areas for attention and change. They attempt quantification, but financial control is not necessarily a strong feature.
Cost/benefit models, such as the one presented in this thesis, are not yet prominent. They seek to establish the financial consequences of maintenance actions and plans, in terms of overall profit to the organisation. Benefit analysis has been established as a little-understood subject in the development world, but for maintenance the problem may be more tractable.
This chapter introduces the author's 7-level model of software maintenance. The purpose of the model is to span a wider range than previously existing models, by considering technical actions and entities at its lower levels and investment factors at the higher ones.
Section 4.2 explains the broad outline of the model and introduces the seven levels. Sections 4.3 to 4.9 then describe the individual levels in more detail.
Section 4.5 gives the overall operation of a maintenance team. It goes into greater detail than the other sections, as this level is used extensively in the later chapters.
Overall, the 7-level model contains more detail than is strictly needed for the thesis. Many of the levels are therefore described only briefly, but the intention is to show that the model would provide a suitable framework for extensions to the present work.
The subject of software maintenance is large and complex. As with any such subject, it is useful to have a simplified model, to act as a focus for discussion. Without such a model:
Such a model is of most help if real maintenance organisations can be seen to map onto it, and should not be judged by the proportion of startlingly new ideas it presents. This model has been produced with that in mind. It may be seen as an extension of the existing models presented in Chapter 3.
The 7-level model breaks the subject area into levels in broad accordance with managerial levels in an organisation. The lowest levels are the most detailed; the higher ones take increasingly broad views of the process until at the very highest, the software being maintained is seen as just one component of a company's asset base.
The levels of the model will first be listed from the highest down, then described from the bottom up. The list is:
Topics are the most detailed elements of the model. At the topic level, individual actions performed by members of the maintenance team are considered. Issues include:
The individual steps of the modification cycle models presented in Chapter 3 would in the 7-level model be considered as topics.
Functions may be considered as groupings of topics, and the following list of functions is representative:
The team level, as its name implies, considers the work of a single maintenance team. It brings together the functions and topics for that team, and shows how they relate and interact.
Figure 4.1 shows the diagram that summarises the actions of the team.
Within the team, these communications are received by the Front Desk duty, which retains records of them. It is the responsibility of the Front Desk to provide answers/solutions, either directly or by passing on the request to a more specialised duty within the team.
The Front Desk has access to a set of known answers/solutions, which is represented in the model as the Repository. This represents information in a variety of forms: versions and variants of the software product(s), paper records of answers to frequently asked questions, etc. Knowledge that resides only in the heads of those performing the Front Desk duty is also regarded as being in this store.
If the solution is in the store or can quickly be generated from information that is, then it is immediately to hand and can be issued back to the customer without further delay. The arrow from the Front Desk to the Delivery Desk and thence out to the customer shows this flow.
Otherwise, a new solution must be generated. We have just accounted for the cases where this can be done quickly, so we now assume that the request must be queued until effort becomes available.
This queue is represented as the Request Store, which contains the backlog of unactioned requests. Customers would like this store to be empty at all times, but economics and staff availability usually dictate otherwise.
The management of the Request Store is an important function. Priorities must be assigned among its contents, and preliminary investigations and impact analysis performed in order to plan future work. The world changes and moves on, so these assessments must be revisited from time to time to keep them current [Fos89].
These preliminary investigations may reveal that the team does not itself have the resources or capability to provide the answer to the problem. For instance, a fault may actually lie in a software module provided (and maintained) by some other team or company. In this and other such cases, a request must be made to that team or company for a solution to be provided, and this team will then be the customer as far as they are concerned. On the diagram, such further teams lie to the right of the boundary rectangle.
If the team can deal with the request, however, then it will be dealt with as one of a repeated series of actions in which the highest priority request is taken from the store and (usually) a software change designed.
The holding store for these changes is the Change Store, into which solutions received from other teams are also placed.
From time to time, the decision will be taken to build a new release of the software, incorporating all new changes available. (There are exceptions to that "all": emergency action may require just a single vital change to skip past the rest).
New releases are then lodged in the Repository, from which they are available for distribution to the original customers.
The team model of figure 4.1 works well at that level, but it is rare for the whole support of a product to be carried out within a single team.
Where more than one team is involved, there will typically be a front-line user support team, which may call on a technical system support team, which may in turn call on separate teams for the different components (modules) of the product as a whole.
This situation is modelled by representing each team as a box after figure 4.1, and showing the boxes joined to indicate the chain of customer/supplier relationships that then exists.
The result is a maintenance channel diagram as shown in Figure 4.2.
At the channel level, we are concerned with studies of the overall response to requests. For example, user satisfaction with product maintenance is greatly affected by response time between original request and installed solution. The channel of teams must itself be studied, as well as the response times of the individual teams. Communications between teams are also studied at this level: requests from one team do not reach the next instantaneously. If the boundary between two teams is also a boundary between two companies, contractual negotiations may increase the delays. Users are concerned only with overall responses, but the channel management must be aware of the details of the channel performance if they are to concentrate efforts where they will do most good.
Most maintenance teams have responsibility for several products, and the competing demands of these products on the team's resources cause interactions between the products.
The network level of the model is concerned with these interactions, and its basis is an augmented version of the channel diagram.
In this augmented version, separate channel diagrams are drawn for each product supported by the team, and these are then overlaid to show the broader network which is the set of teams and customers with which this team will interact and whose operations it will affect. The flow and queueing metrics are then applied to the network, and bottlenecks are identified. As a result, channels may be reorganised and/or resources redistributed between teams.
Another version of the network level considers all products used by a particular customer or group of customers, and draws the augmented network of teams as it affects the customer. This permits the evenness of support across the product range to be examined and if necessary optimised.
At this level, the concern is with the sets of products that support particular business functions of the company. It is valuable to assess and compare:
The benefit of this exercise (which is known as Portfolio Management and can be obtained as a commercial service) is that the evenness of support across business functions can be assessed and compared, and effort directed to where it will best serve the company as a whole.
This is the highest level of the model. It considers the entire set of software products owned by the company, and is concerned with the total value of this investment and the overall costs and paybacks associated with it.
The benefit is that software can then be assessed in terms of its importance to the business as a whole, and ranked accordingly for strategic planning purposes. These include research and development allocations, as well as broad budget decisions.
A question which occurs here, is whether (and if so how) software assets should be represented on the books of the company, alongside its buildings and other visible assets. Some companies have indeed done this, but for most the immediate question is how to derive the appropriate figures rather than what the accountants should do with them next.
A comprehensive model of software maintenance has been presented, in which seven levels of description combine to link the most detailed actions with their financial consequences in terms of investment.
The most important level as far as this thesis is concerned is the team level, which encapsulates the local concerns of a single maintenance team. At this level can be seen the flows of change requests from submission to resolution. Measurements of flow, queue lengths and delays are possible at this level, and can be related to the team level diagram.
The model that lies at the centre of this thesis is presented in detail in this chapter. It is a part of the previously sketched 7-level model, selecting those attributes and functions that are of concern to an individual maintenance team and its immediate management.
The model must, if it is to be successful, assist the maintenance manager who has control of the rest of the team and its own operations, but who must direct those operations in response to outside pressures over which the team can exert little or no control. A related problem faced by the team is that of imperfect information: customers and suppliers are rarely willing to impart all information that could be helpful (usually for commercial rather than technical reasons) and the model must either avoid requiring information which in practice would not be available or have an explicit mechanism for dealing with uncertainty.
The description will first outline the business and numerical assumptions that underlie the model, and which are in effect the criteria that it must meet. The team diagram is then reintroduced in Section 5.4, and the model's components are derived from it. It is shown that the form and use of the team diagram viewpoint permit the collection of metrics that are of value to both the customer and the team itself, and differences between the customer and team viewpoints of these metrics are discussed.
The descriptions of the model's parameters and calculations then form the bulk of the chapter. Each quantity is defined as either a description or an equation; where an equation is not possible, an indication of the algorithm that performs the calculation is given.
The calculation as a whole is necessarily iterative in nature, and the mathematics of continuous behaviour cannot be applied. Section 5.11 discusses the consequences.
Section 5.12 sets out the limitations of the model which might have to be addressed in any practical implementation.
The model to be presented in this chapter takes various input parameters and performs calculations on them to generate intermediate and output results. These quantities are given by definition, calculation and/or equation (as appropriate) in the text, and definitions, calculations and equations share the same numbering scheme.
For convenience, two tables are provided in which all the parameters and results are listed, together with their symbols and the references of their definitions or the equations in which they are defined. Table 5.1 lists the parameters, and Table 5.2 lists the intermediate and output results.
Since it is a requirement (Section 1.2.4) that the model be realistic in accepting basic facts of commercial life, it is necessary first to document the necessary assumptions about those basic facts. There are three:
Stated in such bald terms, this could be the classic description of any get-rich-quick, cowboy operation. But profit is a complex, long-term quantity. Companies which are to be successful in the long term need to spend money on maintaining their reputation, which may include the finance of activities that in themselves make no profit at all; nevertheless, in the long term view, profit is still the underlying motive.
In addition to the recognition of commercial aspects, the model is required to provide quantified predictions for use in decision making. It must therefore deal in numbers and measurements, and have a coherent structure so that different quantities are related and can take part in calculations. Specifically:
Figure 5.1 repeats the team diagram, drawn according to convention with customer(s) on the left and supplier(s) on the right. A general description of its operation has been given in Chapter 4. We will now concentrate on the quantifiable aspects of its use.
There is also considered to be a historic cost attribute (see below), which at the time of receipt is zero.
The token's description attribute is qualitative rather than quantitative. It is assumed that it will be augmented by the team to record actions, opinions and results as the token is processed; but it is not required or used by the model, which deals only with quantified data. The description attribute will therefore be ignored for the rest of this formal discussion.
The historic cost attribute, in contrast, is wholly quantitative. As the token is further processed, the costs of so doing will be added to the attribute, which may be kept either as a simple running total or in the manner of an itemised bill. In further descriptions, this accumulation of value will generally be taken for granted. The historic cost is money that has already been spent, and as such it takes no part in future projections and plans.
A further attribute is known as the event times attribute. Whenever a token passes from one functional area of the team diagram to another, the time of the transition is recorded in this attribute (which therefore has the form of a list of entries). Later, metrics information may be extracted from this list.
Tokens are received by the Front Desk, and there marked with the time of receipt in the event times attribute. Access to the Repository will show whether a solution is immediately available; this will happen if the token is a simple request for information, or if the solution is to provide a more recent release of the software which is already available. In this case, the token passes directly from the Front Desk to the Delivery Desk, and delivery takes place. At delivery, the token is returned to the customer via (2), and its attributes, augmented with the time of delivery, are recorded in the Repository.
No Immediate Solution If no immediate solution was available, the token must next pass from the Front Desk to the Request Store, where it will await attention. At this point, it gains the further attribute of priority, which relates to but is distinct from the importance that was originally present. The priority attribute will be developed further in Section 5.8, but for now it is sufficient to observe that priority will be a scalar quantity which primarily expresses the team's view of the cost-effectiveness of delivering any particular token in the queue. It will also have a secondary function that if a token's priority lies below a given threshold level, the further processing will have been deemed to cost more than it is worth, so unless events cause the priority to be revised upwards, such tokens should never leave the queue unless withdrawn by the customer. There will also be cases in which the calculation of priority becomes irrelevant, because a change is mandated by legislation or other over-riding force. In such cases, there is a mechanism to bypass the normal mechanism and assign an effectively infinite priority to a change. (See Section 5.8.2).
Although priority is a scalar quantity, it is not directly an input to the model but instead the result of a calculation involving costs and benefits. The costs are the estimated costs and delays of completing the design of the change, and are expressed as the change effort and the external change cost.
These components express the expected costs of getting the design as far as the Change Store. We will next introduce two terms which will allow groupings of the components, for which we will need first to define staff cost.
Discussion of benefits follows a similar but more complex mechanism, and will be discussed in Section 5.6.
Management of the Request Store is the next process to affect and be affected by the token and its attributes. Management of the store (as far as the model is concerned) consists of a periodic re-examination of all the tokens in the store, with a re-evaluation and updating of their attributes.
The Design of Change Any token at or near the top of the priority queue is then a candidate for the fix process, which in the familiar view means that a maintainer takes a pending request and designs a software change that will satisfy the request, placing the completed design into the Change Store for later incorporation into a new release. The model actually takes a somewhat different view of this process, though it is to be stressed that the difference does not affect the way that the maintainer works. The problem is that while in principle the maintainer may take a request and carry it through to the design of a change in one smooth action, in practice it doesn't always happen that way. Jobs get interrupted by others of higher urgency; a diagnosis may reach a point where it is discovered that the change must be made in a module that is owned by another maintenance team; it may suddenly be discovered that an error report actually arose from misoperation of the system rather than from anything requiring a software change; etc etc. For all these reasons, it is impractical to have the model assume a smooth, one-way flow of tokens along the line from the Request Store to the Change Store.
Instead, the model takes a view in which tokens are considered to remain in the Request Store until the resolution (withdrawal or design of change) is complete. In this view, the actions of the maintainer add to the information associated with the token, and correspondingly add to the historic cost attribute and (hopefully) steadily decrease the future total change cost. On completion of the design, the information will be sufficient to specify the change exactly for inclusion in a build, and also to specify the unit test to be applied after the build to check that the particular change works. At the same time, the future estimate of the total change cost will become zero, and only at that point is the token transferred from the Request Store to the Change Store. The steady reduction in the total change cost will cause the priority to rise correspondingly, which reflects the intuitive notion that a design once started should probably be continued, but still allows it to be interrupted if another token should gain an even higher priority at any stage.
It is in the design of change that the main people resource limitation occurs, because all activity that contributes directly towards the next release is concentrated here until the build starts. The number of staff available to perform this activity will be important to the model, and when later it refers to staff it will be according to this definition:
External Change Design Designs that reach the Change Store through direct action of the team are not the only possibility. As has been remarked, some changes will be in externally maintained modules, and this is where the team must in its turn become a customer. The identification of a token as being of this type will take place while the token is in the Request Store, and will cause a request to be generated to the appropriate supplier via (3) in Figure 5.1. The eventual arrival of the solution from the supplier via (4) will cause the transfer of the token from the Request Store to the Change Store.
Token Creation Revisited There is an important conceptual point associated with this part of the process, concerning the rule that the team neither creates nor destroys tokens. It has been explained that the customer does create and destroy tokens, and suddenly we have the team acting in turn as a customer -- an apparent inconsistency. We have not so far discussed in any detail the communication channels between customer and team, or between team and suppliers. They have been assumed to be mere pipes or conduits. However, it will be seen that some important measurements are made in those channels, and it is for that reason convenient to give them some structure of their own. Figure 5.2 shows a channel between a team and a supplier, though it could equally have been the channel between customer and team.
Beyond the Change Store As far as the Change Store, tokens move individually from one point to another. From here on, they move in batches as sets of changes are incorporated into new releases. Between the Change Store and the Delivery Desk, the processes of editing the source code, recompilation and testing take place. The exact details will vary from project to project, and are no concern of the model, which summarises the whole process in two parameters: the base build cost and the base build time.
When a build, including its associated testing, is complete, the tokens representing the individual changes incorporated in the release are moved collectively from the Change Store to the Delivery Desk. Here, a fresh attribute known as the customer charge is calculated and added to the token's existing attributes: this represents the amount of money the customer is required to pay for that change. (The calculation of charge is a matter for the commercial agreement between team and customer, and is not specified within the general model). Finally, all attributes are copied from the token and recorded in the Repository, and all attributes except the identity and customer charge are stripped from the token, which is then returned via (2).
The model's strict association of tokens with maintenance actions, its control over the creation and disposal of tokens, and its association of attributes directly with tokens, all make it possible to derive consistent metrics to describe the process as a whole. This section considers the aspects of this that are independent of costs.
First, recall that tokens are created only at a Monitoring Point, and circulate only within the team to the right of that Monitoring Point before being returned to the Monitoring Point for destruction. Without any need to be aware of how the team or its suppliers operate, and with no need to receive data from them apart from the defined data associated with the token itself, the Monitoring Point can determine the rates of flow of requests and solutions, and note the distribution of delay times between requests and solutions. It is further aware at any time of how many tokens are in progress within the team, and from observation of the importance attached by the customer it can present these statistics broken down by that importance.
Within the team, the visibility of flows, delays and queues is far more detailed. For example, the number of tokens in the Request Store is exactly the difference between the inflow from the Front Desk and the combined outflow of withdrawals (back to the Front Desk) and change designs (to the Change Store). The main flow from the Request Store to the Change Store is constrained by available money and effort, and the overall change in performance from any particular improvement or extra resource allocation can be calculated in terms of the resulting change to flow rates, and consequential changes to delays and queue lengths.
We now turn to the consideration of the calculation of benefits. We begin with a fairly simple hypothetical example, and examine the calculation of net benefit from that point of view.
Suppose that the software product is installed at various different sites over some geographical area. Suppose further that a communications module within the software is able to signal to a central engineering point that the failure of a hardware component has been detected, and to request the attendance of an engineer to repair the fault. Finally, suppose that an error within this software is causing occasional false alarms, each of which entails a wasted trip by an engineer. How do we calculate the benefit of correcting this error? Several factors must be taken into account:
Generalising, we will say that each wasted trip in the example is one incident of that particular type. The benefit of making the change will then be the average cost of one such incident, multiplied by the total number of such incidents that would have occurred (within the time max of the model) had the change not been made.
To bring some order to this list of factors, we now introduce Figure 5.3, which depicts many of them in graphical form.
The main solid line shows the projected number of installations, rising from Iinit now to an eventual peak, and with a following plateau after which installations are steadily removed from service until the graph ends at time T max.
(Note that the piecewise linear form of this graph is for explanation only; such simplification is not required by the model). The variation in the number of installations over time will be referred to as the installation profile. It is calculated from three input parameters as described in the following set of definitions.
Working to the right along the X axis, the time at which a solution is delivered to the customer is shown as T R, with a subsequent period of customer acceptance assumed which culminates at time T D. The customer acceptance period is taken to include any time necessary to prepare to begin the distribution of the release.
The model assumes that distribution of the software to the installations in the field begins immediately after the completion of the acceptance period. Therefore, if we denote the distribution start time by T D, we have
Firstly, the upgrade is delivered to the factory or other source of new installations. From time T D onwards, every new installation is assumed to contain the new software. Secondly, a programme of upgrading the I(T D) existing installations commences, achieving a distribution rate of some number of installations per unit time. The progress of the upgrade programme is shown as the "Progress of Distribution" line on the graph, and is complete when I(T D) upgrades have been made, or earlier if the retirement of installations will have reduced the number of upgrades to be made. These latter quantities are defined as follows:
In the above equation, T DONE appears twice: once as the quantity to be calculated, and again as a component of the sum on the right hand side. There is no obvious way to eliminate it from the right hand side, if only because I and RI are non-analytic quantities. Because of that, though, any calculation of the formula must be carried out iteratively. The double appearance of the term then ceases to be a problem.
In Section 5.6, the software change considered was a straightforward one, in which it was clear that the individual saving per incident was relatively easy to estimate in financial terms. Often, this is not the case. If we consider only faults in the software, they may manifest themselves as merely irritating (giving rise to customer complaints, perhaps), or even purely cosmetic, such as a spelling error in a report that is never seen by an end customer. Enhancements can be equally troublesome: how does one put an objective value on a change that simply improves the software's human interface, but that gives no other obvious benefit?
In defining a solution to this question, three guiding principles have been followed. They are:
Here, the problem is to set a value for the saving per incident, where it is known that a number of factors may contribute. Suppose it is believed that a particular change will both reduce the number of customer complaints, and reduce the incidence of incorrect hardware fault diagnosis by the company's own engineers. The question to be answered is: "What level of expenditure in making this change would represent the borderline beyond which the change should not be made?" This is a difficult question in itself, which will be all the worse if it has to be re-evaluated in the future in response to a decision by the company to, say, increase perceived quality by making greater efforts to deal with complaints effectively (meaning that it's then prepared to spend more money per complaint).
However, the question can be broken down into four orthogonal sub-questions:
At first sight, this may not look like much of an advance. One subjective question has been replaced by two questions which are far more objective (questions 1 and 3), but there remain two questions (2 and 4) which are subjective in just the same way as was the original.
However, an important gain has in fact been made. Questions 2 and 4 are no longer linked to the particular change; their answers will apply equally for all change requests which involve those factors. Their answers are now system-wide parameters, and the information associated with the individual change request is now far more tractable.
The number of different factors that are identified in this way is known as the incident type count:
In general, if a total of J different factors have been identified that affect the particular system, we require the setting up of a system-wide vector S1...J, and a similar vector R1...J for each change request token. Entries in R represent observed frequencies of events, while the unit benefits of reducing those frequencies are contained once only in S.
We must now examine more closely Question 1 in the list. It asks for the frequency of the incident "per installation per year", but this form of denominator will not serve in all cases. As examples we may suggest two other possibilities: some proposed changes may have no effect on normal operation, but may reduce the costs associated with each new installation. (Looking ahead to Chapter 8, change request 6 on page 153 is of that type). Alternatively, a change may carry benefit over the remaining years of the system's life, but be of such a nature that the change benefit will be proportional only to the number of years itself.
We now have three examples in the "per what?" category, and more may be added if they are found necessary in particular cases. We will next describe the calculations for those three, which we will call the incident frequency bases. Referring back to Figure 5.3 on page 89:
(Clearly, another example could have been based on the time at which distribution is complete).
The actual algorithm for "per installation per year" will not be presented here.
Our three examples of frequency bases need not be restricted to that number; in general:
For each incident type, we must then specify both the frequency with which it occurs and the base for that frequency.
For any given release time, we can now define the frequency multiplier vector, which will apply across all changes.
We are now in a position to define the gross change benefit BG:
Next, to derive the net change benefit, we need to know the future cost associated with the change. But this is just CT = CX + ECCS, so the net change benefit is represented by
The planning of software releases is a juggling act involving benefits, costs and resources. We have now introduced each of these elements into the model, and must next consider how to specify the optimum release. We wish to maximise the net benefit, within the constraints of available resources.
Constraints might in general apply to many kinds of resource, including direct expenditure (cash flow), available staff effort, and such things as time allocation of specialised facilities. We do not attempt to treat all possibilities in the model; instead we focus on the specific and almost universally observed restriction of available effort (Section 2.2 on page 9).
Our problem, then, is to maximise the net benefit within the constraint of available staff effort. In this section, we discuss how the concept of priority contributes to the process.
The priority of a request for change (a token) represents the team's view of its relative value, compared with all others in the queue. In terms of the finance-driven view of their motivations given on page 65, that translates directly into consideration of which tokens will deliver maximum benefit for minimum expenditure.
If all resources (including effort and money) were freely available, we would not need to discuss priority at all. We would simply include all changes that had net positive benefit in the next release, and with no resource limitations we would be able to issue that release immediately.
In practice, our limiting resource is human effort: we must use that effort to best advantage. If two units of effort are available, we use them better for two fixes at one effort unit each that each benefit us by £5000 than by choosing an alternative fix that takes the two effort units for a net benefit of £9000.
The priority, then, must express the desirability of a change in terms of its net benefit relative to the proportion of the scarce resource that it consumes.
But the benefit and resource expenditure have now been derived, in terms of the parameters of the model and the attributes of the tokens. In fact, the priority of a token is now seen to be simply
As has already been indicated, there are situations where the financial calculation of priority is unnecessary, because other factors make it essential that a particular change is carried out with utmost urgency. These situations include changes mandated by legislation, and in safety-critical systems where the absence of change introduces a safety hazard.
Rather than try to force the model to treat these changes as imperative by choosing some large but finite benefit to accredit to them, we actually go further by allowing their priority to be set to an infinite value directly. Infinity can be a dangerous concept to introduce in a model, but here the only subsequent operation in which it will take part is the sorting of the tokens in the Request Store, and an infinite value then floats naturally to the top.
When more than one token has infinite priority, they will all float to the top of the list but the model defines no ordering among them. This is not a problem: the point is that they must all be selected for the next release, and if they cannot be (i.e. if not enough effort is available) then the release is simply unacceptable as a whole.
The value of a release, as we require it, is generally to be expressed as the return on investment, taking into account all the costs and benefits of that investment. Return on investment can be calculated in many different ways [WDK93], but for the purpose of this thesis a simple calculation will be used. We will take the value as being the difference between the benefits and the costs within the time horizon of the model, and we will ignore any future discounting of the value of money. We observe that in a practical implementation, this formula should be replaced by one that reflects the accounting practices of the organisation involved.
Since we can assume here that we are dealing with a specific projected release time, the calculation of priority for each token can be undertaken, and in the course of that calculation we will discover the net benefit BN (c) associated with any individual token c. We require two further results:
Each of these turns out to be fairly straightforward.
For the list of tokens, we examine those in the Request Store, selecting them in order of priority (highest first) until we cannot select any more within the resources available up to the time of release. (But the model will never extend the selection to include tokens with negative priority). This presupposes, of course, that we have calculated the available resource.
For that, we may need to distinguish between resource types. People's time is the most obvious, but in an exotic application the use of special equipment or the time of one specific expert may be at the highest premium. Here, and in order not to complicate the situation beyond bounds, we will assume that the available effort of the maintenance team is the limiting factor.
The available effort, then, will be the product of the number of maintainers available and the time to the start of the build -- not the time of release. In practice, the number of maintainers can be expected to fluctuate over time, and we may wish to explore the effect of adding people to the team during the course of preparation of the release. We recall the concept of the staff profile, which is the time-varying projected number F (t) of maintenance staff available, and we note that the total available change effort or E(T R) is
Given E and for each change T C, the selection of tokens for the release proceeds as follows:
2. Starting at the head of the list of tokens, accept each token into the change list if and only if it has positive priority, and its acceptance will not cause the sum of change efforts EC for the tokens in the change list to exceed the available change effort E.
As a by-product of this process, any token that is excluded but has infinite priority (see Section 5.8.2) should be reported, since in such a case the release cannot fulfill a basic requirement placed upon it.
There remains to be calculated the cost associated with the release itself, known as the release cost or CR. This is the sum of:
Thus we have the definition of the full release cost CR:
We are now in a position to define the calculation of optimum release time T opt. If we are prepared to iterate to a solution (and we are), the calculation of optimum release time T opt is simply a matter of calculating release values V (t) for a sufficient range of possible release times t, and then selecting the time which offers the highest value. So at last:
Discussion: There are two possible cases in which use of the result of this calculation may need care:
There are questions that might be asked of an implementation of the model, whose answers could be generated automatically if the model possessed simple continuous formulae linking its inputs to its outputs. Continuity would hold out at least the hope that the following operations could be carried out algebraically instead of by trial and error:
Without continuity, the model's outputs will from time to time jump suddenly from one value to another, even in response to arbitrarily small changes in the input values.
Unfortunately, continuity is not a property of the model. There are three reasons.
Firstly, the operation of the model involves a step at which the change requests are sorted into priority order. If two or more priorities are nearly equal, then very small variations in input values can switch the outcome of the sort to a new state.
Secondly, after the sort a cutoff is imposed on the change list according to the effort available. A change is either included or not; arbitrarily small input variations will cause step variations in the content of the list of accepted changes.
Thirdly, the selection of which release is best rests on locating the maximum point on an irregular curve which may have several local maxima at similar heights. Again, arbitrarily small input variations can cause a step variation when a new local maximum becomes the global maximum.
This is unsatisfying, but it arises from the situation being modelled and not from any local limitation of the model itself.
There is one glaring omission in the model, which would in general need to be rectified in practice. It is that the arrival of change requests is assumed to stop after the plans are drawn up: there is no element of prediction for new requests that will arrive before the release is made. An accurate prediction would require a crystal ball beyond our present reach, but knowledge of the past history of arrivals of requests would allow an assumption of future arrivals based on the past rates, costs and benefits. Actual change requests will of course be incorporated into the model as they arrive to see the effect on the plans, but prediction could reveal early the need to build in any required contingency.
There are some simplifications in the model, where complete generality has been sacrificed for the sake of clarity. An example is in the expression of the distribution rate U, which is given as a constant but which could more generally have been made a time-varying function. Equally, the point in time at which new installations are assumed to carry the new release is assumed to be the same as the start of the field upgrade programme. In such cases, the extra calculations would not be severe. The judgement was, however, that the approximations were likely to be more acceptable in practice than the more complicated data entry.
The model assumes that the base build cost, base build time, acceptance cost and acceptance time for a release are all constant values. The system test and acceptance processes will, however, include some component for individual tests of the changes incorporated. If required, the costs and times of these individual tests can be added to the model as extra token attributes, to be incorporated respectively into the calculations of change effort and external change cost.
The model in the form given here calculates the release value as a simple profit value, ignoring both future discounts on the value of money and any assessment of the ratio between profit and investment. Most organisations will normally take both these factors into account when comparing investment opportunities, though exact methods do vary. In order to take into account the preferences of any particular organisation, it will be necessary to rework Equation 5.35 and (for the future value of money) its contributory equations 5.26 and 5.34.
The handling of change requests and the subsequent construction and distribution of a new release of software have been described quantitatively, together with calculations that permit the outcome of actions and decisions to be seen. Effort, time and money are the main quantities that appear in the model, and money its chief guiding concern.
The model's inputs are a set of change requests and their associated data, and a set of more general parameters that include descriptions of the build and distribution environments. The model predicts the financial and other consequences of a decision to release the software earlier or later, and selects the release time which will yield the greatest net benefit.
The model may be used in this way to select the best release time, but it can also quantify the effects of process improvements by altering the values of the parameters that would be affected and seeing the results in terms of the new best plan.
Small changes in the parameters fed to the model may result in large differences in the figures generated. This is an inevitable consequence of the existence of the sorting process by priority among the change requests.
This chapter describes the use of the model in preparation for the chapters on its implementation. Section 6.2 introduces the concept of the baseline plan, which is the result of running the model with best-estimate values. Section 6.3 covers the basic exploration and evaluation of this plan, and Section 6.4 discusses the evaluation of alternative situations and their comparison with the baseline.
Section 6.5 looks at how to cope with uncertainties in input values, and how to tell which are the most sensitive and should therefore be checked most carefully.
Section 6.6 then discusses the formation of alternative plans, based on events that may occur and whose consequences need to be anticipated.
Section 6.7 contains remarks on the possible dominance of certain quantities in the determination of results.
The baseline plan corresponds to the best estimate of future events, and will be the central planning document. Its construction is carried out with the aid of the model and is quite simple in concept: one establishes best-estimate values for all the input parameters of Table 5.1, and one performs the calculations described in Chapter 5. The requirements of Section 5.10 (Equation 5.37) mean that many of the calculations must be iterated many times.
The primary results of the calculations are the values of the optimum release time T opt and the release value V (T opt) at that point. The secondary results are then the change list CL and the release completion time T DONE.
In the absence of other constraints, these results will be accepted as the release to aim for. However, if there are other constraints such as a limit imposed on the release completion time then the baseline release will be selected as that which delivers the best value within the constraints.
The baseline calculations yield one optimum scenario, but all scenarios had to be calculated in order to select it. By examination of these sub-optimal alternatives, it will be possible to see and to comment on whether the baseline plan is clearly the best, or whether any other possibilities are nearly as good. If they are, then the decision is not clear-cut and the fact needs to be reported.
Once the baseline plan has been established, it becomes possible to see the effects of possible changes in the parameter values.
It may be proposed, for example, that a new member be added to the team in order to reduce a backlog of change requests. This is a simple change in the model's staff parameter, in which its value increases by one after some allowance of time for recruitment and perhaps training. This in turn increases the available change effort, which increases the number of changes accepted into the change list. The extra net change benefit from the added changes then adds to the total release value, and from the amount of that increase an assessment can be made of how well justified the extra team member would in fact be.
Probably most alternative plans may be dealt with in a similar manner, by adjusting the appropriate parameters and observing the difference in outcome of the calculations. Some, however, will also involve added constraints on the solution, such as a deadline for the full delivery of the new release to the field. A simple deadline will invalidate any release time that cannot meet it, and reduce the range of the search for the optimum release time. In the most general case, an arbitrary constraint would invalidate some particular subset of the possible release times, and the search for the optimum would proceed with the rest.
Each plan accepts all its parameters as though they were known with perfect accuracy, but of course they are not: they are merely best estimates.
It is therefore prudent to ask whether possible inaccuracies in the estimates would affect the outcome of the plan, and if so to what extent. This is a process well known to designers of electronic circuits, where component values cannot be exactly known and the effects of tolerances must be taken into account.
It is here that the non-linearities inherent in the model have their greatest effect, and they do make the process rather difficult. If, for example, we decide that the value estimated for a particular parameter may be out by 10% either way, we may re-run the model with those boundary values and observe that (say) the plan remains intact. But perhaps a 5% increase in value would have led to a radical change in the plan through crossing one area of non-linearity, only to find the original plan reinstated at 7%.
We will not offer a perfect solution to this problem, though we do note it as a subject for future research. Here, we need observe only that if extreme values do not suggest a change in the plan, the presence of an anomalous region in the unexplored part of the range need not upset us: it would represent an alternative plan which itself could not be relied upon since by its definition the parameter value is not considered reliable enough to guarantee hitting it.
The exploration of stability in practice, then, is a process of varying parameter values to the limits of their likely ranges and checking whether the resulting plan would differ significantly from the baseline. If it does, then the first practical reaction is likely to be to re-examine and improve the estimate in order to reduce its range; if this is not enough, the variation must be noted as a risk that the planned release date (or value, or content) may not be met.
The accuracy of estimated values is one source of variation in the final outcome, but there is another which arises from radical changes in one or more parameters. This is the risk analysis part of the planning process, whose purpose is to test how the plan might be impacted by external events. Small impact means the events would be manageable if they occurred; large impact may imply the need to prepare additional contingency plans.
The specification of what events (risks) should be checked for is a matter of practical judgement, and no model can substitute for that. But the effects of those events are seen as changes in the parameters that are provided to the model, and the impacts on the plan are seen as the consequent changes in the output values. The model thus plays its part in the risk evaluation process.
Risk evaluation, then, is conducted by assessing the new parameter values should the event occur, and re-running the model to observe the effect. On the model, a risk is implemented just as an alternative plan.
The exploration of variations in the input parameters may be expected to reveal that some are more important than others in the determination of the final result: that is, that the values of the less important parameters may vary quite widely without materially affecting the outcome. This is a beneficial effect, for it is what allows attention to be focused on the major parameters in terms of checking and reassessing their likely values.
An example of dominant values will be seen in Chapter 8, where in the list of eight change requests to be studied, four of them generally come out to have much higher priorities than the other four. This usually guarantees their presence at the head of the priority table and in the recommended release.
In general, those changes whose benefits are given in terms "per installation per year" are likely to gain the highest priorities. That is because the frequency multiplier for these is derived from the area under the installations graph and is typically much larger than the alternative multipliers. It seems that an area measure of that sort is not intuitively estimated by people, which can also make the figures generated by the model larger than might otherwise be expected.
At its most basic, the model may be used to predict the outcome of a given maintenance situation. However, its main value is in the ability to explore alternative situations. Input values will be in large part estimates, and by varying their values it is possible to see how uncertainties will affect the main prediction. From that, values whose estimates may need further refinement may be identified.
Future events (risks) that may upset the plan may also be explored, giving an opportunity for contingency plans to be created as and where necessary.
In most situations, there will probably be some parameters whose effect on the outcome is stronger than that of others. Change requests whose impact will be felt in proportion to the number of installations and to the number of years of use are likely to be associated with this effect.
This chapter describes the implementation of the model as a computer program. Section 7.2 describes the development package used, and Section 7.3 describes the implementation in more detail, with full descriptions of the display screens that are seen by its user. Section 7.4 notes some limitations of the implementation.
HyperCardTM is a development package which runs on the Apple MacintoshTM computer, and like the machine itself is well suited to applications with a graphical interface. Applications developed in this way do not stand alone but run under the main HyperCard program. This acts analogously to a conventional language interpreter, though there is a certain level of automatic precompilation.
User applications are referred to as "stacks" because they appear in the manner of a stack of index cards, of which any one can be at the front and therefore visible at any one time. In terms of a more conventional description, cards are alternative screen displays, which may be presented to the user in any desired order and in response to user actions.
Cards may contain two main types of active element: buttons and fields. Buttons are sensitive to mouse clicks from the user, which trigger scripts (program fragments) written as part of the application. Fields may also be sensitive to clicks, but their primary purpose is as holders of fixed or editable text. Artwork may also be drawn on cards.
The scripting language of HyperCard is comprehensive and flexible, but can be slow in heavily iterative applications such as the present model. To get around this, program fragments may also be coded in a conventional high level language (C and Pascal are supported) and included in the application in their object code form. Use has been made of this facility in the implementation.
The model is implemented as one main HyperCard stack and several subsidiary ones. The subsidiary stacks are a stack called "Background Index" to be described first, and an expandable set of "situation stacks" which are cloned from the main one.
The Background Index stack is important to the implementation, but does not implement any central part of the model. As the model runs, cards are created and destroyed as scenarios are calculated or change requests are added and deleted. HyperCard has no built-in mechanism for keeping track of such changing collections of cards, but the model has to maintain lists of what exists and update the lists as changes occur. It is this process that is carried out by the handlers in the Background Index stack.
The situation stacks provide the means whereby alternative situations may be investigated in the model and compared afterwards. The mechanism is simple in concept. After initial data entry and the preparation of the baseline plan, the entire main stack (containing both data and programs) is copied and named. Each time another variant is created, it too is copied and named. The names of these new stacks are collected in a menu available to the user, and comparison of situations is then a matter of switching from one to another by selecting the menu items.
The mechanism of situation stacks is wasteful of space because of the copying of all the program information that takes place. Greater recourse to the "stacks in use" facility would ameliorate the problem, but it was considered acceptable in a prototype system.
The main stack of the model is called "Priority Control" and contains all the meat of the programs and data. There are six backgrounds whose cards the user sees:
The middle four of these will now be illustrated and described.
Figure 7.1 shows a typical summary screen.
The graph eventually reaches a peak, which represents the optimum release value occurring at the optimum release time T opt. An arrowhead marks the peak, with a legend that confirms the time at which it occurs.
After the peak (in this example) there is a steady decline as the benefit of inclusion of any further changes is outweighed by the loss in overall benefit due to the delay in delivery.
The vertical axis of the graph is automatically scaled to accommodate the peak value, which is shown against the axis. This figure must therefore always be checked when comparing different graphs, since two visually similar ones may in fact be drawn to very different scales.
A small triangular marker sits on the time axis, at month 26 in this example. The user can drag it to any month; it sets a limit to the number of scenarios (see below) that will be generated in detail for subsequent examination. Scenario generation is time-consuming, and those well after the peak are of little interest.
At the top right is a square button divided horizontally into three parts. The user clicks on one of these parts to be taken to that section of the model.
To each value of release time on the summary screen (up to the time set by the limit marker) there corresponds one scenario display screen which may be examined for further details of the implications of a release at that time. An example of a scenario display is given as Figure 7.2.
In some cases there will be a vertical dashed line just before the right boundary of this graph. When present, it indicates spare effort which is not enough to accommodate another change into the build.
Starting further up the vertical axis, the number of installations I is then plotted as a function of time. Here, it rises steadily over two years to a maximum of 4700 and then remains constant at that value. The graph is automatically scaled such that its peak uses the full vertical extent available on the display.
Using the same vertical scale, the progress of field updates after the release is then plotted. It is the diagonal line to the right of the release marker. Its start is delayed by the acceptance time A (here two months) so it begins at the distribution start time T D. It then rises at the distribution rate.
Also from the distribution start time, new installations are assumed already to contain the new release. The dashed line which starts on the installations graph at the distribution start time and runs horizontally to the right reflects this. When it intersects with the distribution progress line, distribution is complete. The completion is marked by a vertical dashed line from the meeting point down to the time axis, and a legend below the axis confirms the completion time.
To the right of the composite graph is a table, containing a list of all the outstanding change requests with their assigned priorities. The list is divided into two sections by a pair of rows of asterisks; above these are the changes to be included in the release and below are those rejected. There are two rows of asterisks because of one special case: if a mandatory change cannot be included in the release through insufficient effort being available, it will be listed between the rows. This is a means of flagging what would in practice be a severe headache for the maintenance manager. The table is shown to the right for convenience here, but in the actual implementation it has to overlay part of the composite graph in order to fit within the physical display screen. It is shown or hidden via the "Show RFCs" button below the composite graph.
Below the composite graph is a pair of arrow buttons (with the release time shown between them). These buttons will switch to the previous or next scenario, for comparison of release dates in the form of a slide show. There will sometimes be situations where the optimum release time as calculated will be unacceptable due to other constraints, and it is then necessary to explore other scenarios until the best within the constraints is found. The slide show is convenient at such times, and it may be interspersed with visits to the summary screen by clicking with the mouse anywhere within the main graph area. From the summary screen, any scenario may be visited directly by clicking at or above the appropriate point on the time axis, including on the line of the summary graph itself.
At the bottom left of the display are the primary cost and benefit figures for the scenario. The table shows the release benefit, the release cost, the release value and the return on investment.
Figure 7.3 shows the data entry screen for the general parameters of the model.
The buttons to the right of the scrolling field lead to dialogues which make it easier to set constant values across ranges of months.
Figure 7.4 shows the data entry screen for a change request.
Below this and to the left are entered the short title of the request and a longer description; the description is for illustration but is not otherwise used by the implementation.
To the right of the description are the estimated cost and required effort for the request, and the total cost is calculated by reference to the staff cost parameter entered on the General Parameters screen.
Just below the cost information is the check box (here shown clear) which when checked will mark this request as mandatory.
The lower half of the screen shows the assessed benefits of the change. The second column lists the incident types, and for each type the incident value is shown in the leftmost column. The incident values are shown on each change request screen, though they are the same for all.
The incident frequency values are entered in column 3, and we see that the need for this request comes from the numbers of service interruptions and callouts. With each frequency is associated a frequency base in column 4, given here as a code number to be interpreted according to the small table at the bottom right of the display.
Above this latter table is a button labelled "Edit." This, when clicked, reveals a small display of data editing buttons with instructions for their use.
Firstly, it is unlikely that the HyperCard development system would be used in a practical implementation. Its flexibility and ease of graphic design have made it a good choice for the purposes of this thesis, but there is a penalty in terms of speed of operation. It copes well when asked to deal with a list of only eight change requests, but if that number were to increase perhaps into the hundreds the speed would be unacceptable.
A practical project to which the method was to be applied would in any case be expected already to have its own database system for change control. It would be much more reasonable to expect that the functionality would be added to the existing system, and that the (presumed) existence of a specialised database package would provide the speed to handle the algorithms.
In entering the benefits associated with a change request, the implementation does not permit one incident type to be associated with more than one frequency base. If, for instance, a fault is incurring the same type of cost both per installation per year and per new installation, only one of these can be entered. This restriction allowed the screen display for change requests to be simplified.
The implementation does not allow for the change delay parameter. This would be required in a full implementation.
The model has been implemented on a Macintosh computer, using the HyperCard development system. Alternative plans are created as separate stacks (in the terminology of HyperCard; databases would be an approximately equivalent term). Exploration of plans is a matter of navigating around and between these plans, and navigation controls activated by the mouse are provided and have been explained.
The implementation has four types of display, of which two are used for data entry and two for the presentation of results. The data entry displays are for change requests and for the model's general parameters; there is a main summary screen with a graph of profit against release time, and for each release time examined there is a scenario display showing the detailed composition and outcome of a release timed at that point.
This chapter describes the use of the model with sample data. Although the model has been tested on real projects, there are two reasons why actual data from those projects will not be presented here:
The decision is therefore to base the chapter heavily on one particular project on which the model was tested, but to insert specimen figures that will illustrate the points made. The numbers used in the examples do not therefore reflect actual or typical assessments of costs and benefits. The real project does have in common with the one presented here:
None of these attributes limits the generality of the model, but between them they do permit a wider range of behaviours to be explored.
Sections 8.2 to 8.2.3 set out the baseline against which the subsequent examples are to be compared. Section 8.2 describes the system generally, including the data that will be entered as the general parameters. Section 8.2.2 then presents the change requests that are assumed to be present.
Section 8.2.3 then presents the results of running the model on the sample data. The displays are shown, and a table gives a comparison of the model's conclusions for the range of possible release timings.
Each of Sections 8.3 to 8.7 then considers a variation of the baseline situation, and examines how the baseline plan would be affected.
Section 8.8 presents a summary of the results, comparing the assessments of the different variations.
Our hypothetical system is owned by a (presumably also hypothetical) telecommunications company and forms part of the telecomms network. It is a concentrator, which is the term for a small switch serving perhaps a few hundred telephone customers. The switch handles all local calls within its customer base, but is parented on a larger switch which deals with calls to and from wider areas. The parent switch also handles requests for most specialised services. The concentrator software probably consists of around 100,000 lines of code -- about a quarter of a mile by Foster's metric. Within the concentrator, the software is held in read-only memory on a replaceable circuit board, so installing any upgrade means a site visit and a board replacement.
There are currently 3,500 installations in service, and that number is projected to rise to 4,700 over the next two years. After that, no further units are expected to be installed. None will be taken out of service during the next five years, which is as far ahead as our planning is expected to look.
The software is maintained by one person, who also has another project of similar size to look after. The ongoing effort available for this task is therefore 0.5 of a person. The cost of employment for a full-time person is £60,000 per year.
There are only eight change requests outstanding on the software, which is a small number for such a system but convenient for the purpose of discussion. In partial compensation for that, each request will require quite substantial effort, ranging from 0.5 to 3 person-months. In addition, implementation of each change is associated with a certain amount of direct expenditure in re-issue of operational documentation etc: this ranges from £100 to £5,000.
The build process for such a system may take no more than a day, but the severe consequences of failures will dictate an extended test period before handover. Here, we assume that the build and test will take two months and require total expenditure (including staff time) of £100,000.
After the handover, a further period is required for acceptance testing and a pilot field trial, and for preparing for the full distribution. We assume this process takes a further two months and costs a total of £150,000.
Distribution then begins and it too will take time, dictated partly by the availability of staff to visit the sites and partly by the supply of freshly programmed memory boards. We will assume that in total, it will be possible to upgrade 200 sites each month at a cost of £100 each.
These values are entered on the general parameters screen, as shown in Figure 8.1.
These are the requests:
The data for each request is entered into its own screen. The example used in Chapter 7 (Figure 7.4) in fact shows the data for request number 5: Transmission Loss.
With the above data entered, the model is ready to run and Figure 8.2 shows the summary screen after calculations are complete.
If we wish to examine the slightly lower peak of profit for a release after month 13, we can look at that scenario (Figure 8.4).
By examining each of the scenarios, a full picture may be built up of the implications of all the different possible release dates. On the model the natural way to do this is to use the arrow buttons to move back and forth, but here we will extract the principal items and show them as Table 8.1.
Change 6 ("Database Auto Load") is of value only for new installations, of which there are no more after month 24. For releases after month 11 or 12, enough new installations remain to be done that it gets a high enough priority for inclusion; after that it remains out of the rankings until month 19, when the value (and hence priority) is much reduced but the resource is available to take in lower priority changes. After month 22 there are so few new installations yet to be done that the benefit does not outweigh the cost, and so it drops again out of the list. Change 7 ("One-Step Install") undergoes a similar effect but from a lower base priority; it just makes it into the list after month 21, but after that it descends once more into oblivion.
Clearly, the availability of the maintainer's time is putting severe limitations on the ability to deliver a timely release. Is it worth the extra salary cost of making our maintainer full time?
To examine the effect of taking this action, it is necessary only to alter the "Resource" column in the general parameters screen to show a constant value of 1 (see Figure 8.5).
It may be observed that the appropriate response in fact would be to bring the release forward by another two weeks, since the period during which the resource is spare is not used for any other purpose. That would be done in practice, but the model does not suggest it because one month is its minimum time resolution.
The return on investment for this release is given in Figure 8.7 as 678%, so by this measure also it is an improvement on the original.
As an alternative to the resource increase, we might wish to explore the effect of spending money to improve the test process. The two months required for the build and system test, if it could be reduced, would certainly bring in more benefit through earlier release.
We will suppose that a specialised team can be hired for the system test. Their efforts will reduce the build and test time from the present 2 months to one, but the cost of their service will be £125,000. Should they be used?
In order to investigate this one, we return to the general parameters screen and return it to its baseline state as in Figure 8.1. Then, we add the £125,000 to the release preparation cost and reduce the release preparation time to 1.
Rerunning the model now gives Figure 8.8 as the summary screen.
The decision in this case has been very marginal: it hinges on a difference of only 0.27% in the release value. If we had not made the simplifying assumption of constant value of money but instead assumed a future discount rate of 3% per annum or more, the test team would have been shown to be justified.
Under the assumptions of the baseline plan, no installations of the system are taken out of service during the five-year planning horizon. We now consider what would be the effect of an advance in technology, such that a replacement system renders ours obsolete within the five-year period. We will not be concerned with the economics of the new system: our need is simply to adjust the maintenance plan for the old one, if that should prove necessary.
We will assume that the new system will begin to enter service in two years time, so the installations of our system will be removed progressively from then. Starting in month 24, systems will be replaced at the rate of 125 units per month; from month 36 this figure will increase to 200. At these rates, all installations have been replaced by the end of month 50.
Is it still worth issuing another release of the software, or should we call it quits now?
Working again from the baseline figures, we enter the new data into the model by setting the new values into the retired installations field of the general parameters screen. Again we rerun the model, and we examine the new summary screen (Figure 8.10).
So far, none of the situations we have examined has involved a mandatory change to the software. We will now rectify that.
New EC legislation is being enacted. Three years from now, it will become illegal for any company to ask its workforce to use software that does not communicate with them in their own native language.
Suddenly, the English Commands change takes on a whole new importance. Will this affect the baseline plan, and if so, how?
Returning to the baseline data, we set the "Force Selection" checkbox in the RFC screen for English Commands, and rerun the model. The new summary screen appears as Figure 8.12.
Unfortunately though, the best release does not comply fully with the legislation because the new release will not have been installed at all sites within the required three years. Figure 8.13 shows this scenario, which misses the deadline by four months.
For a typical release situation, more time is spent upgrading the installations in the field than in preparing the changes. This is a consequence of our initial assumption of an embedded system, but it would be expected in practice that distribution arrangements would be carefully reviewed to try and improve the system.
For this example, we will assume that such a review has been carried out. It would indeed be possible to speed things up, but only at a price.
The new scheme would mean assigning extra staff to the distribution task, and purchasing extra spare memory boards. As a result, the distribution rate can be doubled from the baseline figure of 200 installations per month to 400. However, there will be an initial outlay of £50,000 for equipment and training, and each upgrade will cost an additional £50.
These figures all affect the general parameters screen, leaving it as shown in Figure 8.16.
We have seen in the six scenarios how changes in the overall situation can affect the plans for the release of a new version of the software product, and the profit to be made from that release.
Figure 8.19 shows the six outcomes as a scatter plot of release value against optimum release time.
The greatest effect of all, however, is in the allocation of more staff time to the project. It brought the release time forward by a full six months, and yielded a useful increase in profit into the bargain. Discounting of the future value of money would further have increased the value of this situation.
This chapter has shown the use of the model based on hypothetical sample data, which however is based on data from a real project to which the model was applied.
The use of the model to provide a plan for a single situation was first demonstrated. It showed how the priority calculation would distinguish (by negative or positive priority) between those changes that could with advantage be included, but it further showed that positive priority is not in itself enough to gain inclusion in a release. The maintenance effort means that a change must also carry enough benefit to outweigh the release delay while it is designed, and many will not meet this condition.
The further use of the model to explore variations on the baseline situation was then demonstrated. The provision of extra resource for change design provided the greatest advantage, which accords well at least with this author's own experience.
The maintenance of computer software is a process which in an average organisation will consume more than half of the total software budget. Demand (in the form of change requests) normally outstrips supply (in the form of software changes in new releases), and prioritisation is an important component of the process. It is normal practice for organisations to allocate fixed budgets to maintenance activities, within which the target is to implement as many changes as possible from those with the highest priorities.
Existing models of the maintenance process concentrate on actions and entities. The thesis has presented a larger model, where the actions and entities are recognised at the lower levels but higher levels are increasingly concerned with costs, benefits and investment returns.
The thesis is concerned with the planning of future releases of software, and the presented model is used in the derivation of calculations to yield financial implications of different plans. These calculations have been implemented as a computer program.
The program in turn is given a set of sample data to work on, and used to establish a baseline plan for a new release. Variations in the situation are applied, and the resulting effects on the plans are noted.
Assessing the maintenance activity on an investment basis is significantly more difficult than for many other decisions, largely because many individual activities must be taken into account. Each of these has its own cost/benefit attributes, and the calculation of overall plans is repetitive and tedious: in fact, impractical without machine assistance.
The model has been applied primarily to one project within BT, involving an embedded system installed in large numbers within the network. The original figures have not been used in this thesis for reasons of commercial confidentiality, but sample figures have been substituted that allow similar conclusions to be drawn.
Although the model lacks some features that a practical implementation would require, those features would be feasible in a full implementation.
Three criteria for success were stated at the start of this thesis.
1. Describe the maintenance process with the aid of a quantified model that provides financial analysis of the consequences of proposed maintenance actions.
We have developed a 7-level model of the maintenance process, which is described in Chapter 4. Aspects of the model which are relevant to release planning are formalised in Chapter 5. The model predicts the outcome of maintenance situations, in financial terms and with timescales. The model goes beyond the immediate needs of the thesis, and is a suitable basis for further developments. The comparison of alternate investment values will need to be tailored for the organisation that uses the model, and in Section 5.12 we have indicated how this would be done.
2. Show that the model aids decision making both within a maintenance project and in the comparison of investment values between projects.
The model aids decision-making, as introduced in Chapter 6 and as demonstrated by example in Chapter 8. The examples in Chapter 8 illustrate the comparison of alternative situations by reference to their investment returns. Investment returns are absolute financial values, which may be compared amongst different projects including non-software ones. The presentation of information from the model may be improved after further study, and in Section 9.4.2 we indicate the possibility of future work in this area.
3. Show that the model can be implemented on a computer and that the implementation has the potential of practical commercial use.
The model has been implemented in a computer program, and while the current implementation is more suitable as a demonstration than for direct commercial use, we have indicated in Section 7.4 the changes that would be required.
Further research work based on this thesis can be as follows:
Further work to aid the exploitation of the results of this thesis can be as follows:
In conclusion, the following main points have been expanded in this thesis:
[CB86] James S. Collofello and Stephen Bortman. An analysis of the technical information necessary to perform effective software maintenance. In Proc. 5th Annual Phoenix Conference on Computers and Communications, pages 420-424, March 1986.
[Dre92] Daniel W. Drew. Tailoring the Software Engineering Institute's (SEI) Capability Maturity Model (CMM) to a software sustaining engineering organisation. In Proc. 8th IEEE Conf. Software Maintenance, pages 137-144, November 1992.
[RU89] H. Dieter Rombach and Bradford T. Ulery. Establishing a measurement based maintenance improvement program: Lessons learned in the SEL. In Proc. 5th IEEE Conf. Software Maintenance, pages 50-57, October 1989.
base build cost, 46, 58, 76, 77, 80
base build time, 46, 58, 75, 76, 80, 96
change delay, 46, 50, 75, 76, 101
change effort, 46, 50, 53, 54, 70, 72, 76, 81
change list, 47, 74-77, 84, 85
customer charge, 50, 58
description, 50, 51
distribution cost, 47, 76, 77
distribution rate, 46, 64, 80, 96
distribution start time, 47, 63, 64, 68, 69, 77, 96
event times, 50, 52
external change cost, 46, 50, 53, 54, 70, 75, 81
frequency base, 46, 50, 68-70, 100, 101
frequency base count, 47, 69, 70
frequency multiplier, 47, 50, 70, 88
gross change benefit, 47, 50, 70, 71
historic cost, 50-52, 55
identity, 50, 58
importance, 50-52, 59
incident frequency, 46, 50, 67, 69, 70, 100
incident type, 50, 101
incident type count, 47, 67
incident value, 46, 67, 70, 100
initial installations, 46, 62, 63, 98
installation profile, 63, 69
installations, 47, 63-65, 68, 76, 77, 88, 96, 98
net change benefit, 47, 50, 70-72, 74, 77, 85
new installations, 46, 63, 68, 98
optimum release time, 47, 77, 78, 84, 85, 94, 97, 124
priority, 47, 50-53, 55, 71-74, 76
release benefit, 47, 77, 98
release completion time, 47, 64, 65, 84
release cost, 47, 74, 76, 77, 98
release time, 47, 63, 64, 70, 75-77, 85, 93, 95, 97, 124, 125
release value, 47, 73, 74, 77, 78, 81, 84, 85, 93, 94, 98, 115, 124
retired installations, 46, 63-65, 98, 117
return on investment, 98
staff, 46, 55, 56, 75, 85, 96, 98
staff cost, 46, 53, 54, 56, 70, 99
staff profile, 56, 75
time max, 46, 61, 62, 68, 70, 78, 98
time now, 46, 62, 63, 75, 78
total change cost, 47, 50, 54, 55, 70, 71
unit upgrade cost, 46, 76, 77