How Java Programmers Test Exceptional Behavior
Exceptions often signal faulty or undesired behavior; hence, high-quality test suites should also target exceptional behavior. This paper is a large-scale study of exceptional tests—which exercise exceptional behavior—in 1,157 open-source Java projects hosted on GitHub.
We analyzed JUnit exceptional tests to understand what kinds of exceptions are more frequently tested, what coding patterns are used, and how features of a project such as its size and number of contributors correlate to the characteristics of its exceptional tests.
We found that exceptional tests are only 13% of all tests, but tend to be larger than other tests on average; unchecked exceptions are tested twice as frequently as checked ones; 42% of all exceptional tests use try/catch blocks and usually are larger than those using other idioms; and bigger projects with more contributors tend to have more exceptional tests written using different styles.
The paper also zeroes in on several detailed examples involving some of the largest analyzed projects, which refine the empirical results with qualitative evidence. The study’s findings, and the capabilities of the tool we developed to analyze exceptional tests, suggest several implications for the practice of software development and for follow-up empirical studies.
Conference DayMon 17 MayDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:00 - 10:50
|A Traceability Dataset for Open Source Systems|
Mouna HammoudiJOHANNES KEPLER UNIVERSITY LINZ, Christoph Mayr-DornJohannes Kepler University, Linz, Atif MashkoorJohannes Kepler University Linz, Alexander EgyedJohannes Kepler UniversityMedia Attached
|How Java Programmers Test Exceptional Behavior|
Diego MarcilioUSI Università della Svizzera italiana, Carlo A. FuriaUniversità della Svizzera italiana (USI)Pre-print
|An Exploratory Study of Log Placement Recommendation in an Enterprise System|
Jeanderson CândidoDelft University of Technology, Jan HaesenAdyen N.V., Maurício AnicheDelft University of Technology, Arie van DeursenDelft University of Technology, NetherlandsPre-print Media Attached
|Does Code Review Promote Conformance? A Study of OpenStack Patches|
Panyawut Sri-iesaranusornNara Institute of Science and Technology, Raula Gaikovina KulaNAIST, Takashi IshioNara Institute of Science and TechnologyPre-print
|A Replication Study on the Usability of Code Vocabulary in Predicting Flaky Tests|
Guillaume HabenUniversity of Luxembourg, Sarra HabchiUniversity of Luxembourg, Luxembourg, Mike PapadakisUniversity of Luxembourg, Luxembourg, Maxime CordyUniversity of Luxembourg, Luxembourg, Yves Le TraonUniversity of Luxembourg, LuxembourgPre-print Media Attached
|On the Use of Mutation in Injecting Test Order-Dependency|
Sarra HabchiUniversity of Luxembourg, Luxembourg, Maxime CordyUniversity of Luxembourg, Luxembourg, Mike PapadakisUniversity of Luxembourg, Luxembourg, Yves Le TraonUniversity of Luxembourg, LuxembourgPre-print Media Attached
|Discussions and Q&A|