6th Workshop on Automating Test case design, Selection and Evaluation (ATSE)

Based on the submissions we have received this year, ATSE2015 will concentrate on the following topics:
- Learning-based testing
- User experience design and testing
- Model-based statistical testing

The workshop format will be highly interactive! The idea is that the participants COME TO THE WORKSHOP PREPARED and have READ THE PAPERS so that we can have focussed discussions of the topics and to set an agenda and lay the foundation for future development. The papers can be downloaded from the links below in the programme

Program (7th of September)

09:00-9:30 Welcome and Introduction to the workshop

9:30-10:30 Keynote by Joachim Wegener (Berner and Mattner, Germany)
Automatic Generation and Execution of Test Scenarios for Camera-based Driver Assistance Systems

10:30-11:00 coffee break

11:00 - 12:00 Keynote by Cristina Seceleanu (MDH, Sweden)

12:00 - 12:30 Learning-based Testing of Distributed Microservice Architectures: Correctness and Fault Injection (Karl Meinke and Peter Nycander)

12:30-14:00 lunch

14:00 - 14:30 The synergy between user experience design and software testing (Arjan van der Meer, Rachid Kherrazi, Neda Noroozi and Anita Wierda)

14:30 - 15:00 Combining Time and Concurrency in Model-Based Statistical Testing of Embedded Real-Time Systems (Daniel Homm, Jürgen Eckert and Reinhard German)

15:00 - 15:30 Panel discussion on Learning-based testing

15:30-16:00 coffee break

16:00 - 16:30 Panel discussion on User experience design and testing

16:30-17:00 Panel discussion on Model-based statistical testing

17:00-17:30 Putting it all together

17:30 Closure


Joachim Wegener (Berner and Mattner, Germany)

Title: Automatic Generation and Execution of Test Scenarios for Camera-based Driver Assistance Systems

Advanced Driver Assistance Systems (ADAS) are one of the key technologies for future innovations in cars and commercial vehicles. Increasingly, these systems take over safety-related functions, e.g. autonomous braking and autonomous steering. Typically, the systems are based on radar sensors or cameras continuously monitoring the vehicle environment. These sensor technologies cannot provide completely reliable detection rates of objects. Correspondingly, the quality assurance for advanced driver assistance systems is of high importance. In this keynote an approach for automatically testing camera-based driver assistance systems will be presented. Systematic test design and test scenario generation is based on the classification-tree method, environment simulation and image generation are supported using vehicle dynamics and virtual test drive simulations, such as veDyna or CarMaker, test execution and test evaluation are performed using the test environment MESSINA. The proposed test solution enables functional testing of ADAS features in different testing phases, e.g. software in the loop for testing functional models in early development phases, hardware in the loop for testing the electronic control units (ECU) with the integrated software, as well as back to back tests between functional models and ECUs. Additionally, the high degree of automation enables efficient regression testing.

Dr. Joachim Wegener studied computer science at the Technical University Berlin and received his PhD from Humboldt University Berlin in 2001. His thesis work, "Evolutionary Testing of real-time systems' temporal behavior" gained him the "Best Dissertation in Software Engineering" award of the Ernst-Denert-Foundation and the German Computer Society, 2002. Dr. Wegener began his professional career as a scientist at Daimler AG Research and Technology. For DaimlerChrysler research and advanced development he led the software analysis and testing group as well as a group on advanced techniques in software engineering. Since eight years he is working with Berner & Mattner Systemtechnik GmbH - a product and consulting company for embedded systems development. Here, he is responsible for the automotive departments in Berlin, Cologne and Wolfsburg as well as the product development. He is one of the international leading industrial researchers in evolutionary and systematic testing and has published more than eighty papers. He is the inventor of the successful classification tree editor with several thousand users worldwide.

Cristina Seceleanu (MHD, Sweden)

Title: Testing Function and Time for Embedded Systems: From EAST-ADL to Code

Architectural description languages, like EAST-ADL, provide a comprehensive approach when describing complex automotive embedded systems, as standardized models that encapsulate structural, functional, and extra-functional information. The aim of architectural models is to enable the system's documentation, design, early verification, and even code implementation.

In this talk, we show how the use of such models can be extended further, by proposing a methodology that provides code verification based on the EAST-ADL architectural models. Our methodology relies on the automated model-based test-case generation for both functional and timing requirements of EAST-ADL models extended with timed automata semantics, and validation of system implementation by generating Python test scripts from the abstract test-cases. The Python scripts represent concrete test-cases that are then executed on the system implementation. The entire methodology is implemented as a tool chain, consisting of the ViTAL and Farkle tools, and is validated on an industrial prototype, namely the Brake-by-Wire system.


We invite you to submit a paper to the workshop, and present and discuss it at the event itself on topics related to:
  • Formal methods for test cases design, selection, and evaluation (e.g. using theorem proving, rewriting systems, etc.).
  • Formal techniques and tools for automating test case design and selection, e.g. model-based approaches, symbolic-based approach, property based approaches
  • Automated test case design, selection and evaluation from formal models.
  • How formal methods and testing complement each other.
  • Case studies that compare formal approaches to other test case design techniques.

Types of submissions

We invite researchers and practitioners to submit any of the following 3 types of submissions:
  1. Position papers (2 pages) that analyze trends in formal methods and testing and raise issues of importance. Position papers are intended to generate discussion and debate during the workshop, and will be reviewed with respect to relevance and their ability to start up fruitful discussions.

  2. Work-in-progress papers (4 pages) that describe novel, interesting, and highly potential work in progress, but not necessarily reaching its full completion.

  3. Full papers (10 pages) describing original and completed research -- either empirical or theoretical -- in formal methods testing techniques, tools, or industrial case studies.

Co-located at the 13th International Conference on Software Engineering and Formal Methods (SEFM), will be held in York, UK between 7-11 September 2015.


General Chair:
Tanja Vos (Universidad Politecnica de Valencia, ES)

Programme Chairs:
Sigrid Eldh (Ericsson, SE)
Wishnu Prasetya (Universiteit van Utrecht, NL)

Publicity Chair:
Anna Esparcia (Universidad Politecnica de Valencia, ES)

Programme Committee Members:
Pekka Aho (VTT, FI)
Alessandra Bagnato (Softeam, FR)
Maria Alpuente (Universidad Politecnica de Valencia, ES)
Arie van Deursen (TU Delft, NL)
John Derrick (University of Sheffield, UK)
Maria Jose Escalona (Univ. de Sevilla, ES)
Gordon Fraser (Univ. de Sheffield, UK)
Herman Geuvers (Radboud University Nijmegen, NL)
Marieke Huisman (Universiteit van Twente, NL)
Teemu Kanstrén (VTT, FI)
Peter M. Kruse (Berner & Mattner, DE)
Beatriz Marin (Universidad Diego Portales, Chili)
Karl Meinke (KTH, SE)
Mehrdad Saadatmand (MDH, SE)
Marielle Stoelinga (Universiteit van Twente, NL)
Jan Tretmans (TNO, NL)
Javier Tuya (Univ. de Oviedo, ES)
Marc-Florian Wendland (Fraunhofer, DE)
Carsten Weise (imbus AG, DE)
Burkhart Wolff (LRI, FR)

Important dates

Paper Submission: May 22, 2015
Author Notification: June 19, 2015
Camera-Ready: July 3, 2015
Date of Workshop: Sept 7, 2015

Format and submission

All accepted papers will be published as part of a Lecture Notes in Computer Science (LNCS) proceedings. PDF versions of papers should be submitted through the EasyChair submission system using the LNCS Formatting Guidelines. If a paper gets accepted, at least one of the (co)author(s) is expected to be present at the workshop to present the paper. All papers submitted to the workshop must be unpublished original work and should not be under review or submitted elsewhere while being under consideration. Submitted papers will be reviewed by 3 members of Program/Organizing Committee (or their sub-reviewers) and selection of accepted papers will based on relevance, quality and originality of the submitted papers.