Measuring code understandability is essential for the correct maintenance of the quality of a software code base. However, code understandability can be measured only by carrying out empirical studies that involve developers. Therefore, the collected measures concerning the properties of interest depend on the characteristics (e.g., skills and experience) of the involved developers. In this paper, we investigate a method to extract developer-independent data from these empirical studies, so that data can be used to assess the properties of the code with as little noise as possible. We consider a pair of evaluations made by the same developer as the elementary bit of objective evidence. These data are elaborated via the Elo algorithm to rate the understandability of Java methods. The result is a ranking that describes the relative understandability of methods. We applied the proposed method to a code understandability dataset whose analysis did originally yield no appreciable results. When the same type of analysis was carried out on the understandability rating, both correlations with code metrics and accurate understandability models were found. To build models that predict interesting code properties, researchers need to process experimental data so that the considered property depends as little as possible on developers’ performance: this paper proposes a new methodology that addresses this need, thus it is comprised in the scope of the ICSEA 2026 conference topics.

A Method to Extract Developer-independent Information from Data Accounting for Developers’ Performance Variability

Luigi Lavazza;Gabriele Rotoloni;Sandro Morasca
2026-01-01

Abstract

Measuring code understandability is essential for the correct maintenance of the quality of a software code base. However, code understandability can be measured only by carrying out empirical studies that involve developers. Therefore, the collected measures concerning the properties of interest depend on the characteristics (e.g., skills and experience) of the involved developers. In this paper, we investigate a method to extract developer-independent data from these empirical studies, so that data can be used to assess the properties of the code with as little noise as possible. We consider a pair of evaluations made by the same developer as the elementary bit of objective evidence. These data are elaborated via the Elo algorithm to rate the understandability of Java methods. The result is a ranking that describes the relative understandability of methods. We applied the proposed method to a code understandability dataset whose analysis did originally yield no appreciable results. When the same type of analysis was carried out on the understandability rating, both correlations with code metrics and accurate understandability models were found. To build models that predict interesting code properties, researchers need to process experimental data so that the considered property depends as little as possible on developers’ performance: this paper proposes a new methodology that addresses this need, thus it is comprised in the scope of the ICSEA 2026 conference topics.
2026
ICSEA 2026 - The Twenty-First International Conference on Software Engineering Advances
9781685584122
ICSEA 2026 - The Twenty-First International Conference on Software Engineering Advances
Barcelona, Spain
27/9-1/10 2026
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11383/2220352
 Attenzione

L'Ateneo sottopone a validazione solo i file PDF allegati

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact