Abstract:
In a method for determining a quality assessment of a software code, the coverage is concomitantly calculated when determining the assessment. In order to increase the coverage, additional measurement results and assessments may be taken into account. Following changes to the software base, it is determined which of the additional measurements and assessment results should be renewed in order to provide or ensure the defined coverage.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a U.S. National Stage Application of International Application No. PCT/EP2011/067012 filed Sep. 29, 2011, which designates the United States of America, and claims priority to DE Patent Application No. 10 2010 043 623.2 filed Nov. 9, 2010 and DE Patent Application No. 10 2011 006 215.7 filed Mar. 28, 2011. The contents of which are hereby incorporated by reference in their entirety. 
     TECHNICAL FIELD 
     The present disclosure relates to a method and an apparatus for the determination of a quality assessment of a software code with determination of the assessment coverage. 
     BACKGROUND 
     Checking the quality of software entails both a high effort in terms of time and also an associated high effort in terms of cost. In particular, with a manual quality assessment it is frequently impossible to determine reliable, reproducible and/or constructive suggestions for the improvement of the software quality within a framework that is acceptable in terms of both time and money. 
     The information on the quality of software codes obtained by pure benchmark solutions based on metrics or static code analyses is of limited use only. Generally, only lists of metrics with details of threshold crossings or a quantity of rule infringements are generated. A manual analysis and/or assessment of these results is sometimes just as complex as the quality analysis of the software code itself. 
     There is frequently a desire to obtain weighted information for proposed improvements to a software code, which, depending on the effort, costs and expected potential for improvement, enable measures for improvement to be prioritized in order to achieve a desired quality level under prespecified project aims or to approximate them as closely as possible within the limits of the available budget. 
     There is a requirement for a quality analysis for software codes, wherein the results of the quality analysis should enable conclusions to be drawn regarding the aspect or assessment coverage. At the same time, a quality analysis should simultaneously suggest procedures as to how the assessment coverage can be increased under predetermined framework conditions, possibly by means of manual quality analysis. 
     In this context, the assessment must include other aspects in addition to those aspects of the quality analysis for software codes which can be measured by automated means. 
     SUMMARY 
     One embodiment provides a method for determining a quality assessment of a software code and the assessment coverage with the steps: performance of a static code analysis of the software code with the aid of predetermined rule and/or metric definitions and outputting of identified rule infringements and/or object dimensions, which include results of the metrics for software objects; assessment of the identified rule infringements and/or object dimensions based on predetermined assessment functions and the outputting of assessed rule infringements; aggregation of the assessed rule infringements based on a predetermined quality aspect hierarchy and outputting of a quality-aspect-related quality assessment of the software code; and determination of a quality-aspect-related assessment coverage factor based on a predetermined aggregation function and normalization of the identified quality-aspect-related quality assessment to the identified assessment coverage factor for outputting an assessment coverage of the quality-aspect-related quality assessment. 
     In a further embodiment, the method further comprises outputting a multi-quality-aspect-encompassing quality assessment of the software code based on the aggregated assessed rule infringements. 
     In a further embodiment, the method further comprises assessment of an effort required for the rectification of the assessed rule infringements based on predetermined rule properties and the identified object dimensions; and outputting effort-assessed corrective actions. 
     In a further embodiment, the method further comprises sorting the effort-assessed corrective actions according to the amount of effort required and/or severity of the rule infringement; and outputting a group of effort-assessed corrective actions based on predetermined target-achievement data. 
     In a further embodiment, the outputting of assessed rule infringements includes the outputting of assessed rule infringements by software code objects that have changed in comparison with already quality-assessed software code objects. 
     In a further embodiment, the method further comprises determination of quality-assessment tasks, which are to be performed manually, based on the assessment coverage of the quality-aspect-related quality assessment; and outputting an updated assessment coverage taking into account the assessment effort for the identified quality-assessment tasks. 
     In a further embodiment, in the step of the determination of quality-assessment tasks the only quality-assessment tasks to be taken into account are those which are to be performed on software code objects that have changed in comparison with software code objects already quality-assessed in a previous method. 
     Another embodiment provides an apparatus for determining an assessment coverage of a quality assessment of a software code with: a database mechanism, which is embodied to store, predetermined rule and/or metric definitions, predetermined assessment functions, predetermined quality aspect hierarchies, predetermined aggregation functions and predetermined rule properties for a plurality of software codes; an analyzer mechanism, which is embodied to perform a static code analysis of the software code with the aid of predetermined rule and/or metric definitions from the database mechanism and output identified rule infringements and/or object dimensions, which include results of the metrics for software objects; an evaluation mechanism, which is embodied to assess the identified rule infringements and/or object dimensions based on predetermined assessment functions from the database mechanism and output assessed rule infringements; an aggregation mechanism, which is embodied to aggregate the assessed rule infringements based on a predetermined quality aspect hierarchy from the database mechanism and to output a quality-aspect-related quality assessment of the software code; and a normalization mechanism, which is embodied to determine a quality-aspect-related assessment coverage factor based on a predetermined aggregation function and to normalize the identified quality-aspect-related quality assessment to the identified assessment coverage factor for the outputting of an assessment coverage of the quality-aspect-related quality assessment. 
     In a further embodiment, the normalization mechanism is further embodied to output a multi-quality-aspect-encompassing assessment of the software code based on the aggregated assessed rule infringements. 
     In a further embodiment, the apparatus additionally comprises an effort-assessment mechanism, which is embodied to determine an effort required for the rectification of the assessed rule infringements based on predetermined rule properties and the identified object dimensions and to output effort-assessed corrective actions. 
     In a further embodiment, the apparatus additionally comprises a prioritization mechanism, which is embodied to sort the effort-assessed corrective actions according to the effort required and/or the severity of the rule infringement and to output a group of effort-assessed corrective actions based on predetermined target-achievement data. 
     In a further embodiment, the apparatus additionally comprises a selection mechanism, which is embodied to determine quality-assessment tasks, which are to be performed manually, based on the assessment coverage of the quality-aspect-related quality assessment and to output an updated assessment coverage taking into account the assessment effort for the identified quality-assessment tasks. 
     In a further embodiment, the selection mechanism is embodied, during the determination of the quality-assessment tasks to be performed manually, only to take into account the quality-assessment tasks, which are to be performed on software code objects that have changed in comparison with already quality-assessed software code objects. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Exemplary embodiments will be explained in more detail below based on the schematic drawings, wherein: 
         FIG. 1  is a schematic representation of an apparatus for the determination of an assessment coverage of a quality assessment of a software code according to one embodiment; 
         FIG. 2  is a schematic representation of an apparatus for the determination of an assessment coverage of a quality assessment of a software code according to a further embodiment; 
         FIG. 3  is a schematic representation of a mode of operation of an analyzer mechanism according to a further embodiment; 
         FIG. 4  is a schematic representation of a mode of operation of an evaluation mechanism according to a further embodiment; 
         FIG. 5  is a schematic representation of a mode of operation of an aggregation mechanism according to a further embodiment; 
         FIG. 6  is a schematic representation of a mode of operation of an effort-assessment mechanism according to a further embodiment; 
         FIG. 7  is a schematic representation of a mode of operation of a prioritization mechanism according to a further embodiment; and 
         FIG. 8  is a schematic representation of a mode of operation of a selection mechanism according to a further embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     Certain embodiments of the present disclosure perform a fully automatic source-code evaluation based on expanded quality models. In this context, based on the results of a static code analysis, predetermined aggregation functions are used which enable the determination of the degree to which the static code analysis covers all aspects of the quality assessment of the software code. This enables conclusions to be drawn from the relative results of static code analysis regarding the absolute information or coverage content of the quality assessment. 
     Therefore, one embodiment provides a method for the determination of an assessment coverage of a quality assessment of a software code, with the steps of the performance of a static code analysis of the software code with the aid of predetermined rule and/or metric definitions and outputting identified rule infringements and/or object dimensions, of the assessment of the identified rule infringements and/or object dimensions based on predetermined assessment functions and outputting assessed rule infringements, of the aggregation of the assessed rule infringements based on a predetermined quality aspect hierarchy and outputting a quality-aspect-related quality assessment of the software code and of the determination of a quality-aspect-related assessment coverage factor based on a predetermined aggregation function and normalization of the identified quality-aspect-related quality assessment to the identified assessment coverage factor for outputting an assessment coverage of the quality-aspect-related quality assessment. 
     According to one embodiment, the method can comprise the steps of outputting a multi-quality-aspect-encompassing quality assessment of the software code based on the aggregated assessed rule infringements. 
     According to one embodiment, the method can comprise the steps of the assessment of an effort required for the rectification of the assessed rule infringements based on predetermined rule properties and the identified object dimensions and of outputting effort-assessed corrective actions. 
     According to one embodiment, the method can comprise the steps of sorting the effort-assessed corrective actions according to the effort required and/or severity of the rule infringement and of outputting a group of effort-assessed corrective actions based on predetermined target-achievement data. 
     The outputting of assessed rule infringements can comprise an outputting of assessed rule infringements by software code objects that have changed in comparison with already quality-assessed software code objects. 
     According to one embodiment, the method can comprise the steps of the determination of quality-assessment tasks, which are to be performed manually, based on the assessment coverage of the quality-aspect-related quality assessment and of the outputting of an updated assessment coverage taking into account the assessment effort for the identified quality-assessment tasks. 
     In this context, it can be provided that only those quality-assessment tasks are taken into account which are to be performed on software code objects that have changed in comparison with already quality-assessed software-code objects. 
     Other embodiments provide an apparatus for the determination of an assessment coverage of a quality assessment of a software code. The apparatus comprises a database mechanism, which is embodied to store predetermined rule and/or metric definitions, predetermined assessment functions, predetermined quality aspect hierarchies, predetermined aggregation functions and predetermined rule properties for a plurality of software codes. The apparatus further comprises an analyzer mechanism, which is embodied to perform a static code analysis of the software code with the aid of predetermined rule and/or metric definitions from the database mechanism and to output identified rule infringements and/or object dimensions. The apparatus further comprises an evaluation mechanism, which is embodied to assess the identified rule infringements and/or object dimensions based on predetermined assessment functions from the database mechanism and output assessed rule infringements. The apparatus further comprises an aggregation mechanism, which is embodied to aggregate the assessed rule infringements based on a predetermined quality aspect hierarchy from the database mechanism and output a quality-aspect-related quality assessment of the software code. The apparatus further comprises a normalization mechanism, which is embodied to determine a quality-aspect-related assessment coverage factor based on a predetermined aggregation function and normalize the identified quality-aspect-related quality assessment to the identified assessment coverage factor for the outputting of an assessment coverage of the quality-aspect-related quality assessment. 
     According to one embodiment, the normalization mechanism can further be embodied to output a multi-quality-aspect-encompassing quality assessment of the software code based on the aggregated assessed rule infringements. 
     According to one embodiment, the apparatus can further comprise an effort-assessment mechanism, which is embodied to determine an effort required for the rectification of the assessed rule infringements based on predetermined rule properties and the identified object dimensions and output effort-assessed corrective actions. 
     According to one embodiment, the apparatus can further comprise a prioritization mechanism, which is embodied to sort the effort-assessed corrective actions according to the effort required and/or severity of the rule infringement and output a group of effort-assessed corrective actions based on predetermined target-achievement data. 
     According to one embodiment, the evaluation mechanism can be embodied to output assessed rule infringements of already quality-assessed changed software-code objects. 
     According to one embodiment, the apparatus can also comprise a selection mechanism, which is embodied to determine quality-assessment tasks which are to be performed manually based on the assessment coverage of the quality-aspect-related quality assessment and to output an updated assessment coverage taking into account the assessment effort for the identified quality-assessment tasks. 
     In this context, the selection mechanism can be embodied to take into account during the determination of the quality-assessment tasks to be performed manually, only the quality-assessment tasks which are to be performed on software code objects that have changed in comparison with software code objects already quality-assessed in a previous method. 
     The above embodiments and developments can, insofar as is meaningful, be combined with each other in any suitable manner. Further possible embodiments, developments and implementations include combinations, not explicitly cited, of features described herein with respect to various features of the embodiments. 
     A source code within the meaning of this disclosure can be any program code written in a programming language which, following compilation, results in executable software. In this context, the source code can be written in any programming language, for example C/C++, C#, Java or similar programming languages. It is also possible for the source code to be present as an intermediate code, as for example, in Java, .NET or a similar format. 
       FIG. 1  shows a schematic representation of an apparatus  10  for the determination of an assessment coverage of a quality assessment of a software code. The apparatus  10  comprises a database mechanism  11 , an analyzer mechanism  12 , an evaluation mechanism  13 , an aggregation mechanism  14  and a normalization mechanism  15 . 
     The following explains the functions and modes of operation of the individual components of the apparatus  10  with respect to the depictions in  FIGS. 3 to 5 , which, for purposes of better understanding, show detailed depictions of the analyzer mechanism  12 , the evaluation mechanism  13  and the aggregation mechanism  14 . 
     As shown in  FIG. 3 , the analyzer mechanism  12  comprises a static code analyzer  21 , which can comprise a scanner  22  and a parser  23 . The static code analyzer accepts a source code  16 , which can be processed in the scanner  22  and/or the parser  23  so that after parsing by the parser  23  code objects are available to which an automatic static code analysis can be applied. 
     To this end, the analyzer mechanism  12  can comprise a metric-calculating device  25  and a rule-evaluating device  24 . In addition to the code objects from the source code  16  parsed by the parser  23 , the metric-calculating device  25  receives as input parameters predetermined metric definitions  11   b . In the metric-calculating device  25 , the predetermined metric definitions  11   b  can be applied to the code objects in order to receive object dimensions  25   a  as the output. The object dimensions  25   a  can in this context in particular comprise results of the metrics for software objects, for example the number of code lines, the number of object linkages, the frequency of executed loops or similar metrics results. In this context, the metrics are not defined as specific metrics and it is possible for any metric definitions  11   b  to be predetermined and evaluated with the metric-calculating device  25 . 
     The analyzer mechanism  12  can also comprise a rule-evaluating device  24 , which receives a predetermined rule definition  11   a  as an input parameter and based on the code objects from the source code parsed by the parser  23  compiles rule infringements  24   a . In this context, the rule infringements  24   a  can be present as an unsorted and unweighted list, which only lists rule infringements  24   a  determined with reference to the predefined rule definitions  11   a.    
     The rule infringements  24   a  and the object dimensions  25   a  can be further used as output values  12   a  of the analyzer mechanism  12  by the apparatus  10 . For the static code analysis in the analyzer mechanism  12 , it is possible, for example, to use freely available tools such as, for example, FxCop, Gendarme, PMD, CheckStyle, sppcheck, SourceMonitor or commercial products such as, for example, Klocwork, Coverity, Sotograph, Understand or PC-Lint. However, obviously, it is also possible to use other tools for the static code analysis in the analyzer mechanism  12 . 
     The predetermined rule definitions  11   a  and the predetermined metric definitions  11   b  can be filed in a database mechanism  11  of the apparatus  10 , as shown in  FIG. 1 . To this end, the database mechanism  11  can comprise the rule definitions  11   a  and metric definitions  11   b  as part of a predetermined quality model, which is used for the quality assessment of software codes. 
     The apparatus  10  further comprises an evaluation mechanism  13 , as shown in  FIG. 4 . The evaluation mechanism  13  receives as input the rule infringements  24   a  and the object dimensions  25   a . The evaluation mechanism  13  also receives one or more evaluation functions  11   c  from the database mechanism  11  based on which the evaluation mechanism  13  assesses the rule infringements  24   a  according to a prespecified assessment system. For example, it is possible to use a point system in order assign malus points to the rule infringements  24   a  depending upon the severity and/or influence. In this context, the object dimensions  25   a , which can, for example, comprise an object list with metrics, can also be used in the assessment of the rule infringements. The evaluation mechanism  13  is designed to output a list with assessed rule infringements  13   a , for example a list with rule infringements  13   a  to which an awarded score is assigned with the aid of the evaluation functions  11   c.    
     The evaluation mechanism  13  can also receive as input manually assessed rules  17   a  and a list with changed software-code objects  17   b , which can be included in the assessment. For example, manual assessments of rules or rule infringements can exert an influence on the assessed rule infringements  13   a . It can also be possible for the evaluation mechanism  13  also to include rule infringements  24   a  as devalued in the assessment of the rule infringements if the manually assessed rules  17   a  relate to changed software-code objects  17   b , i.e. to software-code objects  17   b , which have been subject to changes in comparison to the original software code  16 . 
     The apparatus  10  comprises an aggregation mechanism  14 , as shown in  FIG. 5 . The aggregation mechanism  14  accepts the assessed rule infringements  13   a  from the evaluation mechanism  13  and groups them according to aspects of the quality model. To this end, the aggregation mechanism  14  can receive an aspect hierarchy  11   d  and an aggregation function  11   e  as input parameters from the database mechanism  11 . The aggregation mechanism  14  can use the aggregation function  11   e  to determine for every aspect of the quality model according to the aspect hierarchy  11   d  the rule infringements  24   a  for which points are to be deducted according to the list of the assessed rule infringements  13   a.    
     In this context, the aggregation mechanism  14  takes into account not only those components of an aspect of the aspect hierarchy  11   d  which can be automatically measured with the aid of metrics and/or rule definitions, but also the other components. In this way, in addition to the information on how high the quality of the software code  16  is assessed with respect to automatically measurable quality criteria according to the assessed rule infringements  13   a , it is also possible to assess how high the assessment coverage of a particular quality aspect is. This means for example that, for a quality aspect, a total of 100 points can be issued, but the automatic quality measurement is only able to cover 40 points. In the above example, this results in an assessment coverage degree of 40%. If now, for example, due to point deductions by assessed rule infringements  13   a , a score of 20 is determined by the aggregation mechanism  14 , although a target achievement score of 50% is obtained for the achieved score with respect to the covered 40 points, a target achievement score of only 20% is achieved with respect to the overall possible score of 100. 
     It can for example be provided that, for a better overview, the aggregation mechanism  14  contains a scale of marks according to which, depending on the degree of points achieved, marks, like, for example, school marks, can be assigned. In the above example, the software code  16  in the selected aspect of the aspect hierarchy  11   d  would receive a school grade 4 for a point achievement degree of 50%. Since, however, the automatic code analysis can only cover 40% of the total possible number of points, a school grade of 6 would be obtained for a coverage-corrected grade of the selected aspect. 
     The aggregation mechanism  14  is designed to output the identified quality-aspect-related quality assessments according to a normalization mechanism  15 , which in turn determines a quality-aspect-related assessment coverage factor based on the predetermined aggregation function  11   e  and normalizes the identified quality-aspect-related quality assessment to the identified assessment coverage factor for the outputting of an assessment coverage of the quality-aspect-related quality assessment. In this context, the assessment coverage comprises an aspect assessment  15   a  relating to each individual assessed aspect of the aspect hierarchy  11   d  and an overall marking  15   b  of all aspects of the quality model. On the evaluation of the markings  15   a  and  15   b  output by the normalization mechanism, it can be identified to what degree coverage gaps in the assessment coverage have occurred in the automatic code analysis. At the same time, the result is corrected by the manually performed rule evaluations  17   a  and the corresponding corrections in changed software objects  17   b.    
     The aggregation mechanism  14  can also output a list  14   a  of rule infringements, wherein details of how many points for the infringement have been deducted for which aspect of the aspect hierarchy  11   d  are stored in the list  14   a  for each rule infringement. This makes it possible to identify the proportion of rule infringements both for each individual aspect and for the overall marking. 
     The list  14   a  of rule infringements can be forwarded according to an effort-assessment mechanism  21 , which is embodied to determine an effort required for the rectification of the assessed rule infringements  13   a  based on predetermined rule properties  11   f  and the identified object dimensions  25   a , as shown in  FIG. 6 . It can be provided that the effort-assessment mechanism  21  receives the rule properties  11   f  as input parameters from the database mechanism  11 . With reference to the object dimensions  25   a , this enables the assessment of an effort which will probably be required for the rectification of a rule infringement. For example, it is possible to file as a rule property for each rule infringement details of whether the rectification of the rule infringement necessitates local, module-wide or system-wide changes to the software code  16 . In the case of a local change to the software code  16 , it is possible, for example, to use the number of locally affected code lines, as determined according to the object dimensions  25   a , for the calculation of effort. 
     For example, a local rule infringement can be assessed with a constant for effort, a module-wide rule infringement with a value in dependence on size of the module and a system-wide rule infringement in dependence on the size and number of the modules used. The effort-assessment mechanism  21  can be embodied to output effort-assessed corrective actions  21   a.    
     As shown in  FIG. 7 , the effort-assessed corrective actions  21   a  can be passed to a prioritization mechanism  22 , which is embodied to sort the corrective actions according to their severity. This enables the probable effort identified for each corrective action to be taken into account. The purpose of the prioritization mechanism  22  is to prioritize corrective actions which promise the highest benefit with the lowest effort higher than corrective actions promise a lower benefit or a require a greater effort. To this end, the prioritization mechanism  22  can receive as input parameters effort threshold values  18   a  and quality threshold values  18   b . The effort threshold values  18   a  and the quality threshold values  18   b  are project-related target-achievement data which disclose criteria for the maximal possible effort to be exerted or minimum quality desired. For example, the effort threshold can be achieved if the effort available in terms of costs or time is used up by the corrective actions to be performed. The quality threshold can be achieved if the performance of the corrective actions causes the quality assessment to rise over a certain predetermined amount. 
     The prioritization mechanism  22  can be embodied to generate as output a sorted list  22   a  of suggested corrective actions to be performed. 
     The apparatus  10  can also comprise a selection mechanism  23 , which can be embodied to determine, quality-assessment tasks, which are to be performed manually, based on the assessment coverage of the quality-aspect-related quality assessment and to output an updated assessment coverage taking into account the assessment effort for the identified quality-assessment tasks. To this end, as shown in the diagram illustrating the mode of operation of the selection mechanism  23  in  FIG. 8 , the selection mechanism  23 , can receive as input changed software-code objects  11   b . In a method step  31 , the selection mechanism  23  selects the assessment tasks that have to be performed manually for each object type in order to achieve the required assessment coverage. Here, the rule definitions  11   a  of the quality model can be taken into account as a basis. 
     With reference to the rule properties  11   f , in a step  32 , following the assessment tasks to be performed manually, an effort can be calculated, which will probably be necessary for the assessment tasks to be performed manually. Then, taking into account the aspect assessment with the effort estimation  11   h , the aggregation functions  11   e and the object dimensions  25   a , the evaluation function  11   c  can be used to calculate an effort coverage coefficient in a step  33  which indicates the amount of additional effort with which a corresponding assessment coverage can be increased. 
     All the suggested assessment tasks to be performed manually are sorted in a step  34  in order to generate a list  34   a  with assessment tasks to be sorted manually, i.e. a list  34   a  indicating which assessment tasks to be performed manually promise the highest increase in assessment coverage for the least effort. 
     If in step  35 , it should be identified during a comparison with project-related target coverage specifications  11   g  that the present assessment coverage does not correspond to the target coverage specifications  11   g , a certain number  23   a  of manual assessment tasks to be performed according to the sorted list  34   a  can be suggested to increase the assessment coverage. However, if the target coverage has already been reached, the selection mechanism  23  can stop its work in a step  36 . 
     In some embodiments, the selection mechanism  23  only uses assessment-neutral criteria such as the assessment coverage and assessment effort to be achieved in order to influence a subsequent manual assessment as little as possible.