A Large-Scale Longitudinal Study of Flaky Tests

Lam, Wing; Winter, Stefan; Wei, Anjiang; Xie, Tao; Marinov, Darko; Bell, Jonathan

Author:

Lam, Wing [DBLP] ;

Winter, Stefan [DBLP] ;

Wei, Anjiang [DBLP] ;

Xie, Tao [DBLP] ;

Marinov, Darko [DBLP] ;

Bell, Jonathan [DBLP]

Abstract

Flaky tests that non-deterministically pass or fail without any code changes constitute an impediment to regression testing. To understand when and how flaky tests can be detected most efficiently, we analyzed the commit histories of known flaky tests. We find that 75% of flaky tests are flaky when added, indicating substantial value for developers to run detectors specifically on newly added tests. The percentage of flaky tests that can be detected early increases to 85% when detectors are run on both newly added and directly modified tests.

Citation
BibTeX

Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D. & Bell, J., (2022). A Large-Scale Longitudinal Study of Flaky Tests. In: Grunske, L., Siegmund, J. & Vogelsang, A. (Hrsg.), Software Engineering 2022. Bonn: Gesellschaft für Informatik e.V.. (S. 57-59). DOI: 10.18420/se2022-ws-018

@inproceedings{mci/Lam2022,
author = {Lam, Wing AND Winter, Stefan AND Wei, Anjiang AND Xie, Tao AND Marinov, Darko AND Bell, Jonathan},
title = {A Large-Scale Longitudinal Study of Flaky Tests},
booktitle = {Software Engineering 2022},
year = {2022},
editor = {Grunske, Lars AND Siegmund, Janet AND Vogelsang, Andreas} ,
pages = { 57-59 } ,
doi = { 10.18420/se2022-ws-018 },
publisher = {Gesellschaft für Informatik e.V.},
address = {Bonn}
}

	Dateien	Groesse	Format	Anzeige
	A1-18.pdf	254.4Kb	PDF	View/Open

Sollte hier kein Volltext (PDF) verlinkt sein, dann kann es sein, dass dieser aus verschiedenen Gruenden (z.B. Lizenzen oder Copyright) nur in einer anderen Digital Library verfuegbar ist. Versuchen Sie in diesem Fall einen Zugriff ueber die verlinkte DOI: 10.18420/se2022-ws-018

Haben Sie fehlerhafte Angaben entdeckt? Sagen Sie uns Bescheid: Send Feedback