How Java Programmers Test Exceptional Behavior
Exceptions often signal faulty or undesired behavior; hence, high-quality test suites should also target exceptional behavior. This paper is a large-scale study of exceptional tests—which exercise exceptional behavior—in 1,157 open-source Java projects hosted on GitHub.
We analyzed JUnit exceptional tests to understand what kinds of exceptions are more frequently tested, what coding patterns are used, and how features of a project such as its size and number of contributors correlate to the characteristics of its exceptional tests.
We found that exceptional tests are only 13% of all tests, but tend to be larger than other tests on average; unchecked exceptions are tested twice as frequently as checked ones; 42% of all exceptional tests use try/catch blocks and usually are larger than those using other idioms; and bigger projects with more contributors tend to have more exceptional tests written using different styles.
The paper also zeroes in on several detailed examples involving some of the largest analyzed projects, which refine the empirical results with qualitative evidence. The study’s findings, and the capabilities of the tool we developed to analyze exceptional tests, suggest several implications for the practice of software development and for follow-up empirical studies.
Mon 17 MayDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:00 - 10:50
Testing and code reviewTechnical Papers / Data Showcase / Registered Reports at MSR Room 2
Chair(s): Jürgen Cito TU Wien and Facebook
|A Traceability Dataset for Open Source Systems|
Mouna Hammoudi JOHANNES KEPLER UNIVERSITY LINZ, Christoph Mayr-Dorn Johannes Kepler University, Linz, Atif Mashkoor Johannes Kepler University Linz, Alexander Egyed Johannes Kepler UniversityMedia Attached
|How Java Programmers Test Exceptional Behavior|
Diego Marcilio USI Università della Svizzera italiana, Carlo A. Furia Università della Svizzera italiana (USI)Pre-print
|An Exploratory Study of Log Placement Recommendation in an Enterprise System|
Jeanderson Cândido Delft University of Technology, Jan Haesen Adyen N.V., Maurício Aniche Delft University of Technology, Arie van Deursen Delft University of Technology, NetherlandsPre-print Media Attached
|Does Code Review Promote Conformance? A Study of OpenStack Patches|
Panyawut Sri-iesaranusorn Nara Institute of Science and Technology, Raula Gaikovina Kula NAIST, Takashi Ishio Nara Institute of Science and TechnologyPre-print
|A Replication Study on the Usability of Code Vocabulary in Predicting Flaky Tests|
Guillaume Haben University of Luxembourg, Sarra Habchi University of Luxembourg, Luxembourg, Mike Papadakis University of Luxembourg, Luxembourg, Maxime Cordy University of Luxembourg, Luxembourg, Yves Le Traon University of Luxembourg, LuxembourgPre-print Media Attached
|On the Use of Mutation in Injecting Test Order-Dependency|
Sarra Habchi University of Luxembourg, Luxembourg, Maxime Cordy University of Luxembourg, Luxembourg, Mike Papadakis University of Luxembourg, Luxembourg, Yves Le Traon University of Luxembourg, LuxembourgPre-print Media Attached
|Discussions and Q&A|
Go directly to this room on Clowdr