Andreas Spillner - Software Testing Foundations, 4th Edition [2014, PDF, ENG]

1 Introduction 1
2 Fundamentals of Testing 5
2.1 Terms and Motivation ...................................................................................... 6
2.1.1 Error, Defect, and Bug Terminology ........................................... 7
2.1.2 Testing Terms ...................................................................................... 8
2.1.3 Software Quality .............................................................................. 11
2.1.4 Test Effort ........................................................................................... 13
2.2 The Fundamental Test Process ................................................................... 17
2.2.1 Test Planning and Control ............................................................ 19
2.2.2 Test Analysis and Design .............................................................. 22
2.2.3 Test Implementation and Execution ........................................ 25
2.2.4 Test Evaluation and Reporting ................................................... 28
2.2.5 Test Closure Activities .................................................................... 30
2.3 The Psychology of Testing ........................................................................... 31
2.4 General Principles of Testing ....................................................................... 33
2.5 Ethical Guidelines ............................................................................................ 35
2.6 Summary ............................................................................................................ 36
3 Testing in the Software Life Cycle 39
3.1 The General V-Model ...................................................................................... 39
3.2 Component Test .............................................................................................. 42
3.2.1 Explanation of Terms ...................................................................... 42
3.2.2 Test objects ........................................................................................ 43
3.2.3 Test Environment ............................................................................ 43
3.2.4 Test objectives .................................................................................. 46
3.2.5 Test Strategy ...................................................................................... 48
3.3 Integration Test ................................................................................................ 50
3.3.1 Explanation of Terms ...................................................................... 50
3.3.2 Test objects ........................................................................................ 52
3.3.3 The Test Environment .................................................................... 53
3.3.4 Test objectives .................................................................................. 53
3.3.5 Integration Strategies .................................................................... 55
3.4 System Test ........................................................................................................ 58
3.4.1 Explanation of Terms ...................................................................... 58
3.4.2 Test Objects and Test Environment .......................................... 59
3.4.3 Test Objectives ................................................................................. 60
3.4.4 Problems in System Test Practice .............................................. 60
3.5 Acceptance Test ............................................................................................... 61
3.5.1 Contract Acceptance Testing ...................................................... 62
3.5.2 Testing for User Acceptance ........................................................ 63
3.5.3 Operational (Acceptance) Testing ............................................. 64
3.5.4 Field Testing ...................................................................................... 64
3.6 Testing New Product Versions .................................................................... 65
3.6.1 Software Maintenance ................................................................... 65
3.6.2 Testing after Further Development .......................................... 67
3.6.3 Testing in Incremental Development ...................................... 68
3.7 Generic Types of Testing ............................................................................... 69
3.7.1 Functional Testing ........................................................................... 70
3.7.2 Nonfunctional Testing ................................................................... 72
3.7.3 Testing of Software Structure ..................................................... 74
3.7.4 Testing Related to Changes and Regression Testing ......... 75
3.8 Summary ............................................................................................................. 76
4 Static Test 79
4.1 Structured Group Evaluations .................................................................... 79
4.1.1 Foundations ...................................................................................... 79
4.1.2 Reviews ................................................................................................ 80
4.1.3 The General Process ....................................................................... 82
4.1.4 Roles and Responsibilities ............................................................ 86
4.1.5 Types of Reviews .............................................................................. 88
4.2 Static Analysis ................................................................................................... 95
4.2.1 The Compiler as a Static Analysis Tool ..................................... 97
4.2.2 Examination of Compliance to Conventions 
and Standards ................................................................................... 97
4.2.3 Execution of Data Flow Analysis ................................................ 98
4.2.4 Execution of Control Flow Analysis ........................................... 99
4.2.5 Determining Metrics .................................................................... 100
4.3 Summary ......................................................................................................... 102
5 Dynamic Analysis – Test Design Techniques 105
5.1 Black Box Testing Techniques .................................................................. 110
5.1.1 Equivalence Class Partitioning ................................................. 110
5.1.2 Boundary Value Analysis ............................................................ 121
5.1.3 State Transition Testing .............................................................. 128
5.1.4 Logic-Based Techniques (Cause-Effect Graphing 
and Decision Table Technique, Pairwise Testing) ............. 136
5.1.5 Use-Case-Based Testing ............................................................. 141
5.1.6 General Discussion of the Black Box Technique ................ 145
5.2 White Box Testing Techniques ................................................................ 145
5.2.1 Statement Testing and Coverage ........................................... 146
5.2.2 Decision/Branch Testing and Coverage ............................... 148
5.2.3 Test of Conditions ........................................................................ 151
5.2.4 Further White Box Techniques ................................................ 159
5.2.5 General Discussion of the White Box Technique .............. 160
5.2.6 Instrumentation and Tool Support ........................................ 160
5.3 Intuitive and Experience-Based Test Case Determination ............ 161
5.4 Summary .......................................................................................................... 164
6 Test Management 169
6.1 Test Organization ......................................................................................... 169
6.1.1 Test Teams ...................................................................................... 169
6.1.2 Tasks and Qualifications ............................................................. 172
6.2 Planning ........................................................................................................... 174
6.2.1 Quality Assurance Plan ............................................................... 174
6.2.2 Test Plan ........................................................................................... 175
6.2.3 Prioritizing Tests ............................................................................ 177
6.2.4 Test Entry and Exit Criteria ........................................................ 179
6.3 Cost and Economy Aspects ...................................................................... 180
6.3.1 Costs of Defects ............................................................................. 180
6.3.2 Cost of Testing ............................................................................... 181
6.3.3 Test Effort Estimation .................................................................. 184
6.4 Choosing the Test Strategy and Test Approach ................................ 184
6.4.1 Preventative vs. Reactive Approach ...................................... 185
6.4.2 Analytical vs. Heuristic Approach ........................................... 186
6.4.3 Testing and Risk ............................................................................ 187
6.5 Managing The Test Work ........................................................................... 189
6.5.1 Test Cycle Planning ...................................................................... 189
6.5.2 Test Cycle Monitoring ................................................................. 190
6.5.3 Test Cycle Control ......................................................................... 192
6.6 Incident Management ................................................................................ 192
6.6.1 Test Log ............................................................................................ 193
6.6.2 Incident Reporting ....................................................................... 193
6.6.3 Defect Classification .................................................................... 195
6.6.4 Incident Status ............................................................................... 197
6.7 Requirements to Configuration Management .................................. 200
6.8 Relevant Standards ...................................................................................... 202
6.9 Summary ......................................................................................................... 203
7 Test Tools 205
7.1 Types of Test Tools ....................................................................................... 205
7.1.1 Tools for Management and Control of 
Testing and Tests .......................................................................... 206
7.1.2 Tools for Test Specification ....................................................... 209
7.1.3 Tools for Static Testing ............................................................... 210
7.1.4 Tools for Dynamic Testing ......................................................... 211
7.1.5 Tools for Nonfunctional Test .................................................... 216
7.2 Selection and Introduction of Test Tools ............................................. 218
7.2.1 Cost Effectiveness of Tool Introduction ............................... 219
7.2.2 Tool Selection ................................................................................ 220
7.2.3 Tool Introduction .......................................................................... 221
7.3 Summary ......................................................................................................... 223
A Test Plans According to IEEE Standard 829-1998 225
Test Plans According to IEEE Standard 829-2008 231
B Important Information about the Syllabus and the Certified Tester Exam 241
C Exercises 243
Glossary 247
Literature 277
Index 283