Timothy C. Lethbridge and Nicolas Anquetil

School of Information Technology and Engineering

150 Louis Pasteur, University of Ottawa

K1N 6N6 Canada

tcl@site.uottawa.ca, anquetil@csi.uottawa.ca

We apply coupling and cohesion metrics to various decompositions of a large telecommunications system. Several findings emerge: 1) There is a baseline level of coupling and cohesion that represents the average connectedness of any pair of files in the system. 2) As would be expected, measures of cohesion are greater than this baseline in well-defined subsystems. 3) Interestingly, and contrary to intuition, measures of coupling for such subsystems tend also to be greater than the baseline and to increase as cohesion increases. 4) It is possible to easily calculate upper bounds for coupling and cohesion that no subsystem in a given system can exceed. 5) Measuring quality by subtracting coupling from cohesion, as has been proposed in the literature, gives anomalous results since coupling and cohesion are frequently not on the same scale. We propose improved coupling, cohesion and quality metrics that normalize for the baseline and ceiling levels in a given system. Using our proposed metrics it should be possible to compare different systems, something that the current metrics do not permit.

The general objective of our research is to assist software engineers to understand very large software systems that are typically poorly documented and organized. Part of our approach to this problem involves automatically extracting subsystems—Software engineers will use the extracted subsystems (also known as ‘clusters’) to guide their exploration of an unfamiliar system. For related research see Lakhotia (1997) and Müller et al (1993).

In order to validate our subsystem extraction methods, we wish to evaluate them for quality. We hope to show that the extracted subsystems in some sense encapsulate some useful aspect of the architecture. One approach to such evaluation is to look at coupling and cohesion. In this paper we discuss the use of coupling and cohesion metrics from the literature. We explain several problems with them, particularly the fact that calcuated values do not fall within the same range from system to system.

The next section describes methods we have been using for calculating cohesion and coupling, and a derived measure of quality that is the difference between cupling and cohesion. The subsequent section describes experiments we have performed using these metrics. The remainder of the paper describes problems with the conventional metrics, and proposes improved, normalized, metrics.

We use the method for calculating coupling and cohesion described in Patel et al (1992) and Kunz and Black (1995). The only difference is that in our case we are clustering files, whereas Kunz and Black were clustering ‘processes’. The basis for this method is a function Sim_W(X,Y) that takes two files X and Y, computes vectors X_W and Y_W from these (described below), and computes a value between 0 and 1:

Several different types W of vectors can be used in the similarity functions; these are described in the next section.

The more similar two characteristic vectors, the more closely related their files are likely to be. We would expect a cohesive subsystem to have all its files closely related. Thus Kunz and Black describe their metric for cohesion as the average similarity of all distinct pairs of files in a cluster P:

Here, m is the number of files in the cluster P, and each p is a member of cluster P.

Coupling is correspondingly defined as the average similarity of all pairs of files in the system, such that one file in each pair is in the cluster P, and the other file is outside the cluster:

Here, m is the number of files in cluster P, and n is the number of files not in cluster P.

A good subsystem is expected to exhibit high cohesion and low coupling. However, because there are two separate metrics, comparison between subsystems is difficult. If one subsystem has higher coupling (better) and higher cohesion (worse) than the other, it is not clear which one is the best. Kunz and Black defined a metric for the quality of a system as simply the difference between the cohesion and coupling:

For the method described above, the similarity measure Sim_W(X,Y), depends on how the characteristic vectors are constituted. For our experiments, we used three different methods which we are diffentiating using the W subscript in the above equations:

TR: A characteristic vector counts the references to particular user-defined types within the file it describes. This is the method used by Kunz and Black (1995).

DR: A vector represents the use of named variables in the two files being compared, as defined in Patel et al (1992).

RC: A vector represents the calling of routines in one file by routines in the other and vice-versa. We introduced this method to see whether different ways of computing cohesion and coupling affect our results.

We performed our experiments on a telecommunications system consisting of about 4500 files. We used the 11 techniques described in table 1 to cluster these files into subsystems; our original objective was to evaluate the effectiveness of these techniques at extracting highly cohesive subsystems which were as loosly coupled with each other as possible. However, in the context of this paper, we became more interested in evaluating the cohesion and coupling metrics themselves.

clustering technique clustering technique clustering technique

clustering technique

where is the cohesion of the entire system, i.e. the average of the similarity between all pairs of files. Random subsystems will tend to have:

Column 2 of table 4 shows the value of B for each similarity method. The B values can also be seen in figure 1 where the coupling curves are slightly above the baseline in all three cases. Note that it is possible for Coh and Coup to have values below the baseline. If Coh were below the baseline, it would be indicative of an utterly uncohesive subsystem — worse than in the random case. It would clearly show good design if Coup were to be below the baseline in a subsystem, however in our system it rarely occurs that the average subsystem has this property.

We will use the B values to help derive a new metric in section 4.3.

For any given system and similarity method there will be maxima that the cohesion and coupling of no subsystem can exceed.

We can calculate the cohesion ceiling thus:

In other words, the highest similarity between any pair of files in the entire system. It should be clear that no average of similarities, and hence no cohesion metric for a cluster, could ever exceed this. In the system we studied, always equals 1, since there is always at least one pair of files that have identical similarity vectors. This is shown in column 3 of table 4.

The cohesion ceiling is also an upper bound on the coupling, however an improved upper bound can be defined as:

This is the largest coupling value for all one-file clusters. No larger cluster could have a larger coupling since the calculation of Coup for such a larger cluster would necessarily include file clusters that have a lower Coup than that which was found to have .

Values of for our experimental system are shown in column 4 of table 4. Note that , especially for similarity method RC, they are very low. Note also that the graphs in figure 1, approach these calculated upper bound figures.

As with the baseline metric, we will use the and values in the derivation of new metrics in the next section.

The following summarizes problems identified so far with the metrics described in the literature::

We can solve both of the above problems by normalizing the cohesion and coupling metrics so that the zero point corresponds to the baseline B, and the 1-point corresponds to the appropriate ceiling C. Thus, the normalized cohesion and coupling metrics can be calculated as follows:

Both these metrics have an upper bound of 1, and a theoretical lower bound of -B/(C-B). For cohesion, the practical lower bound will be zero: if the cohesion were below zero then we have an extraordinarily bad subsystem that is less cohesive than a random system. For coupling it is possible to have values that are slightly below zero, but well above -1.

Figure 3 shows three graphs of normalized coupling and cohesion in our experimental system, one for each of the similarity methods.

clustering technique clustering technique clustering technique

The effect of normalization becomes even more prominent when we calculate the new quality metric thus:

Experimental values of NQ are shown in figure 4. When compared with figure 2, two things become apparent: The two curves have common features, with high points at clustering technique 5 and 9, plus low points at clustering techniques 6 and 10. When one looks at the values, however, there is a dramatic change from what Q considered to be the best and worst quality versus what NQ considers best and worst: For example, Q considered technique 1 to be the worst, followed by technique 2; NQ considers technique 6 to be the worst, followed by technique 3.

clustering technique

We can draw the following conclusions from the experiments and new metrics presented in this paper, to the extent that the large system with which we experimented is characteristic of systems in general:

Now that we have presented these results for one software system, the most important future work will be to replicate them with other systems. Another possible research direction might be to experimentally modify the weight of NCoup and NCoh (from the current equal weighting) to discover a quality metric that corresponds best with software engineers’ subjective opinions about quality.