D7.4: Maintainability models of heterogeneous programming models

So far we have examined how performance and energy consumption changed if a program was executed on dierent platforms. Although it is a very important aspect of a program, there are other factors that have to be taken into consideration. In this document we describe how the maintainability of REPARA-C++ source code can be measured and how it changes during the transformation for execution on target devices.

The international standard ISO/IEC 25000:2014  Systems and software Quality Requirements and Evaluation (SQuaRE) [4]  denes the term software maintainability and it determines its five subcharacteristics (analysability, modiability, modularity, reusability, and testability) that inuence it. The standard, however, does not dene how the subcharacteristics can be calculated for a given software or how they should be aggregated to get the maintainability of the software. Because of this, we involved our software engineering experts to build a maintainability model. This means that each expert determined the relative importance of the subcharacteristics and we aggregated their opinions to get a model that calculates maintainability from its subcharacteristics. We used REPARA-C++ source code metrics (defined in D3.6 [1]) to determine the maintainability subcharacteristics. More precisely, each expert selected the metrics that inuence the given subcharacteristics and determined how much they do so. We aggregated these results and thus we obtained a model which indicates how software maintainability (and its subcharacteristics) can be calculated from REPARA-C++ metrics.

In this model, a REPARA-C++ metric represents a metric as an abstract denition but its value cannot be used directly because dierent metrics can have values on very dierent scales (e.g., McCabe’s Complexity is usually between 1 and 100 while lines of code can be in the thousands) and the model would give misleading results. To solve this problem we used the benchmarks, which are our training set, to calculate the empirical cumulative distribution (ECD) function for each metric and we used these functions to transform the metric values into the [0,1] interval. This way they became comparable and the transformed values could be used in the maintainability model. To apply these theoretical results in practice, we developed a Maintainability Assessment Framework (MAF) that calculates the maintainability value for a
given program. More precisely, its input is a set of benchmarks and it rst invokes RCPP2AIR (see D2.2 [5]) to analyse their source code and to create the corresponding REPARA-AIR representations. Next, an improved version of AIR2Metrics is applied which calculates kernel and function level source code metrics. In the next step, these kernel and function level metrics are aggregated to system level and the Normalizer tool calculates the necessary ECD functions and transforms the metrics into the [0,1] interval. Finally, the ModelEvaluator tool gets the maintainability model as an input, which is given in a simple text le in human readable format, and calculates the maintainability metrics for each input system.

In order to examine how software maintainability changes during source code transformation, we have to measure the maintainability of the software before and after the transformation. For this purpose, we established the sequential implementations of the used benchmarks as the original (before transformation) state and the OpenCL variants as the after transformation state. By comparing the system level maintainability values it can be seen that in most cases the maintainability of the transformed systems is worse. On the other hand, the inuence to the kernel level maintainability is ambiguous because the number of cases where the maintainability increased or decreased are almost the same. Since we could not measure the effect of real source code transformation yet, these results have to be handled carefully. However, these results support the usefulness of source code transformation because it is worth developing a more maintainable sequential code and then have it automatically transformed to a target hardware to achieve better performance.


Comments are closed.