2022-05-19 –, Main Hall
Many test suites test both too much and too little. Too much, since some tests are redundant (but still expensive to execute and maintain). Too little, since important functionality remains untested. In this talk, I present test intelligence analyses that harness data from the development process to help you optimize your Robot Framework tests.
Many teams have to test more and more functionality in less and less time. Historically grown test suites often don't cope well with this, since they test both too much and too little. Too much, since they contain tests that are expensive to execute (and maintain) but add little value over similar tests in the same test suite. Too little, because important functionality remains untested.
In the talk, I will present analysis techniques that reveal such problems in your own test suites:
- Test gap analysis shows, which code changes are still untested.
- Test impact analysis and pareto-optimization of test suites identify tests that can find new bugs faster than other tests in the suite.
All analyses work on data from the system under test, such as change information from the version control system and test-wise coverage from the automated tests.
For each analysis, I outline the underlying research and present examples from practice. For some analyses, I use the Robot Framework itself as example, for others I use customer system or data from our own software development process.
Elmar works both as a researcher and a founder. Elmar wrote his PhD thesis on static code analysis and is still active as a researcher in software quality analysis. In 2009, he co-founded CQSE GmbH and since helps teams in using analysis tools more effectively. Elmar frequently talks at research conferences (e.g. ICSE, ICPC) and industry events (e.g. GTD, OOP, JAX). He was elected best speaker at Clean Code Days, Software Quality Days and Java Forum Stuttgart. Elmar was named Junior Fellow of the GI in 2015.