Table of Contents

1. Introduction ……………………………………………………………………………….. 13
1.1. The QuadREAD Project…………………………………………………………………13
1.2. Requirements metamodel………………………………………………………………15
1.3. Problem statement ……………………………………………………………………..16
1.4. Research objective ……………………………………………………………………..16
1.5. Research method ………………………………………………………………………..18
1.6. Contributions …………………………………………………………………………….19
1.7. Document structure……………………………………………………………………..19
2. Background and related work ……………………………………………………….. 21
2.1. Introduction ………………………………………………………………………………21
2.2. Software requirements ………………………………………………………………..22
2.3. Software requirements specifications ……………………………………………24
2.4. Software requirements management……………………………………………..25
2.5. System maintenance engineers ……………………………………………………26
2.6. Change scenarios ………………………………………………………………………..26
2.7. Change impact predictions ………………………………………………………….27
2.8. Requirements models and relations ……………………………………………..32
2.9. Software tools ……………………………………………………………………………34
2.10. Validation approaches ………………………………………………………………44
2.11. Conclusion ………………………………………………………………………………49
3. Experimental design ……………………………………………………………………. 51
3.1. Introduction ………………………………………………………………………………51
3.2. Goal ………………………………………………………………………………………..51
3.3. Hypothesis ……………………………………………………………………………….51
3.4. Design ……………………………………………………………………………………..52
3.5. Parameters ………………………………………………………………………………..53
3.6. Variables ………………………………………………………………………………….54
3.7. Planning …………………………………………………………………………………..56
3.8. Participants ……………………………………………………………………………….61
3.9. Objects …………………………………………………………………………………….62
3.10. Instrumentation ………………………………………………………………………..64
3.11. Data collection ………………………………………………………………………..71
3.12. Analysis procedure ………………………………………………………………….71
3.13. Validity evaluation ………………………………………………………………….72
3.14. Conclusion ……………………………………………………………………………..74
4. Execution …………………………………………………………………………………… 75
4.1. Introduction ………………………………………………………………………………. 75
4.2. Sample ……………………………………………………………………………………… 75
4.3. Preparation ……………………………………………………………………………….. 75
4.4. Data collection performed ………………………………………………………….. 78
4.5. Validity procedure ……………………………………………………………………… 78
4.6. Conclusion ……………………………………………………………………………….. 79
5. Analysis ……………………………………………………………………………………… 81
5.1. Introduction ………………………………………………………………………………. 81
5.2. Change scenario representativeness …………………………………………….. 81
5.3. Golden standard reliability ………………………………………………………….. 82
5.4. Precision-Recall and ROC graphs ……………………………………………….. 86
5.5. One-way between-groups ANOVA ………………………………………………. 86
5.6. Non-parametric testing ……………………………………………………………….. 91
5.7. Analysis of covariance ……………………………………………………………….. 94
5.8. Multivariate analysis of variance ………………………………………………….. 95
5.9. Conclusion ……………………………………………………………………………….. 96
6. Interpretation ………………………………………………………………………………. 97
6.1. Introduction ………………………………………………………………………………. 97
6.2. Change scenario representativeness …………………………………………….. 97
6.3. Golden standard reliability ………………………………………………………….. 97
6.4. Precision-Recall and ROC graphs ……………………………………………….. 99
6.5. One-way between-groups ANOVA ………………………………………………. 99
6.6. Non-parametric testing ……………………………………………………………….. 99
6.7. Analysis of covariance ……………………………………………………………… 100
6.8. Multivariate analysis of variance ………………………………………………… 100
6.9. Conclusion ……………………………………………………………………………… 101
7. Conclusions and future work ………………………………………………………. 103
7.1. Summary ……………………………………………………………………………….. 103
7.2. Results …………………………………………………………………………………… 104
7.3. Limitations ……………………………………………………………………………… 104
7.4. Future work ……………………………………………………………………………. 106
8. Glossary ……………………………………………………………………………………. 109
9. References ………………………………………………………………………………… 113
A. Interviews …………………………………………………………………………………. 119
A.1. Introduction ……………………………………………………………………………. 119
A.2. Goal ………………………………………………………………………………………. 119
A.3. Preparation ……………………………………………………………………………….. 119
A.4. Execution …………………………………………………………………………………. 120
A.5. Information systems academic ……………………………………………………. 120
A.6. Industry experts at Capgemini …………………………………………………… 122
A.7. Conclusions …………………………………………………………………………….. 125
B. Tasks ………………………………………………………………………………………. 127
B.1. Introduction …………………………………………………………………………… 127
B.2. Warming up (REQ_BDS_007) ………………………………………………….. 127
B.3. Task 1 (REQ_PHN_001) ………………………………………………………….. 127
B.4. Task 2 (REQ_SPM_004) …………………………………………………………. 127
B.5. Task 3 (REQ_MAP_002) …………………………………………………………. 127
B.6. Task 4 (REQ_NAV_003) ………………………………………………………….. 128
B.7. Task 5 (REQ_TOR_001) ………………………………………………………….. 128
C. Group matching ………………………………………………………………………. 129
C.1. Introduction …………………………………………………………………………… 129
C.2. Coding ………………………………………………………………………………….. 129
C.3. Pre-experiment randomized ………………………………………………………. 130
C.4. Pre-experiment tuned………………………………………………………………. 131
C.5. Experiment final ……………………………………………………………………… 132
D. Golden standards ……………………………………………………………………. 133
D.1. Introduction …………………………………………………………………………… 133
D.2. Task 1 (REQ_PHN_001) ………………………………………………………….. 133
D.3. Task 2 (REQ_SPM_004) …………………………………………………………. 135
D.4. Task 3 (REQ_MAP_002) …………………………………………………………. 137
D.5. Task 4 (REQ_NAV_003) ………………………………………………………….. 139
D.6. Task 5 (REQ_TOR_001) ………………………………………………………….. 142
E. Box plots ……………………………………………………………………………….. 145
E.1. Introduction …………………………………………………………………………… 145
E.2. Task 1 (REQ_PHN_001) ………………………………………………………….. 146
E.3. Task 2 (REQ_SPM_004) …………………………………………………………. 147
E.4. Task 3 (REQ_MAP_002) …………………………………………………………. 148
E.5. Task 4 (REQ_NAV_003) ………………………………………………………….. 149
E.6. Task 5 (REQ_TOR_001) ………………………………………………………….. 150
F. Precision-Recall and ROC graphs ……………………………………………… 151
F.1. Introduction …………………………………………………………………………… 151
F.2. Legend …………………………………………………………………………………. 151
F.3. Task 1 ………………………………………………………………………………….. 152
F.4. Task 2 ………………………………………………………………………………….. 153
F.5. Task 3 …………………………………………………………………………………… 154
F.6. Task 4 …………………………………………………………………………………… 155
F.7. Task 5 …………………………………………………………………………………… 156
G. WASP requirements ……………………………………………………………….. 157
G.1. Introduction …………………………………………………………………………… 157