REFSQ 2011 » Empirical Studies (Live Experiment)

Empirical Studies (Live Experiment)

REFSQ’11 offers a place for empirical research. As part of the Empirical Track the Empirical Studies contain a live experiment in which everyone can participate. Below you will find more information about the experiment and about the Empirical Studies in general.

Live Experiment about Risk Estimation

The live experiment will be a controlled experiment about the estimation of IT-related risk. Such risk estimation is the basis for risk-based requirements prioritization, for such requirements which contribute to the system value by reducing risk.

Research questions: The experiment investigates the influence of the following two factors on the quality of risk estimation:

Fuzziness: Are the estimates achieved for measurable risks better than those for fuzzy or abstract risks?
Scales: Are damage and probability of risks easier to estimate on a scale of points than in percent or Euro?

Participants´ benefits: The participants can make a practical experience with the estimation of requirements-related risk, what is more difficult than they probably expect.

Profile of intended participants: No special knowledge or experience is needed. Former experiments have indicated that more experience does not automatically mean better risk estimates.

Agenda of the experiment:

14:00-14:20 introductory presentation
14:20-14:50 risk estimation on questionnaires
14:50-15:10 break
15:10-15:30 discussion of preliminary results and method

Study organizer:

Andrea Herrmann, Axivion GmbH

Replication Package:

This is the replication package for the risk-estimation experiment executed and published at REFSQ 2011.

The ppt slides are the introduction presentation.

Each participant gets the following material, depending on whether (s)he belongs
to Group 1 or Group 2:

For typing in the data from the questionnaires – or alternatively to execute
the experiment on computers – you can use the excel templates:

for Group 1/2 respectively.

Call for Participation

The discussion at recent REFSQs have confirmed the strong need for empirical validation of the effectiveness for our RE methods, but the literature to date, including that of REFSQ, could show more of this validation. This lack is assumed to be at least partly due to the difficulty of finding and persuading the participation of a sufficient number of suitable experimental subjects. Therefore, REFSQ 2011 will offer an opportunity to conduct a small number of empirical studies during the conference itself. The goals of this opportunity, besides that of permitting the conduct of some experiments, are to raise awareness for the necessity and benefits of empirical studies and to show that participating in them is not dangerous to one’s health. Furthermore, we want to bring together the community of researchers and practitioners who are interested in empirical studies.

Therefore, we support empirical studies to be conducted during the REFSQ conference. Possibilities include and are not restricted to

a controlled experiment, designed to require no more than 90 minutes
a series of interviews, each interview shall be designed to require no more than 30 minutes
an in-situ survey (questionnaire), designed to require no more than 30 minutes

The featured studies are designed to be performed during conference itself with REFSQ participants from industry and academia volunteering to be subjects. The results of the study will be published one month after the conference!

More information regarding the controlled experiment about the estimation of IT-related risk can be found here.