To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

In our previous article, we have discussed Software Testing Types.

For many projects (though not all) the system is eventually released into the live environment. Hopefully, once deployed, it will be in service as long as intended, perhaps for years or decades.

During this deployment, it may become necessary to change the system. Changes may be due to:

• Additional features being required.
• The system being migrated to a new operating platform.
• The system being retired—data may need to be migrated or archived.
• Planned upgrade to COTS-based systems.
• New faults being found requiring fixing (these can be ‘hot fixes’).

Once changes have been made to the system, they will need to be tested (retesting), and it also will be necessary to conduct regression testing to ensure that the rest of the system has not been adversely affected by the changes. Testing that takes place on a system which is in operation in the live environment is called maintenance testing.

When changes are made to migrate from one platform to another, the system should also be tested in its new environment. When migration includes data being transferred in from another application, then conversion testing also becomes necessary.

As we have suggested, all changes must be tested, and, ideally, all of the system should be subject to regression testing. In practice, this may not be feasible or cost-effective. An understanding of the parts of the system that could be affected by the changes could reduce the amount of regression testing required. Working this out is termed impact analysis, i.e. analyzing the impact of the changes on the system.

Impact analysis can be difficult for a system that has already been released. This is because the specifications may be out of date (or non-existent), and/or the original development team may have moved on to other projects, or left the organization altogether.

You may follow the complete series of Software Life Cycles articles here:

Software Testing Development Models
Test Levels
Software Testing Types
Maintenance Testing

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

In our previous article, we have discussed Software Test Levels.

Test types fall into the following categories:

• Functional testing
• Non-functional testing
• Structural testing
• Testing after code has been changed.

To facilitate different types of testing, models may be used as follows:

• Functional testing: process flows; state transition models; security threat models; plain language specifications.
• Non-functional testing: performance model; usability model.
• Structural testing: control flow model; menu structure model.

Functional Testing

As you saw in the section on system testing, functional testing looks at the specific functionality of a system, such as searching for flights on a website, or perhaps calculating employee pay correctly using a payroll system. Note that security testing is a functional test type. Another type of functional testing is interoperability testing—this evaluates the capability of the system to interact with other specified components.

Functional testing is also called specification-based testing: testing against a specification.

Non-Functional Testing

This is where the behavioral aspects of the system are tested. As you saw in the section on system testing, examples are usability, performance under load and stress, among others. As for functional testing, these requirements are usually documented in a functional specification. Thus, mainly black-box testing techniques are used for this type of testing.

These tests can be referenced against a quality model, such as the one defined in ISO 9126 Software Engineering—Software Product Quality.

Structural Testing

This type of testing is used to measure how much testing has been carried out. In functional testing, this could be the number of functional requirements tested against the total number of requirements.

In structural testing, we change our measure to focus on the structural aspects of the system. This could be the code itself, or an architectural definition of the system. We want to do this to check the thoroughness of the testing carried out on the system that has actually been built. A common measure is to look at how much of the actual code that has been written has been tested.

Note that structural testing can be carried out at any test level.

Testing Related to Changes

The previous sections detail the testing to be carried out at the different stages in the development life cycle. At any level of testing, it can be expected that defects will be discovered. When these are found and fixed, the quality of the system being delivered can be improved.

After a defect is detected and fixed the changed software should be retested to confirm that the problem has been successfully removed. This is called retesting or confirmation testing. Note that when the developer removes the defect, this activity is called debugging, which is not a testing activity. Testing finds a defect, debugging fixes it.

The unchanged software should also be retested to ensure that no additional defects have been introduced as a result of changes to the software. This is called regression testing. Regression testing should also be carried out if the environment has changed.

Regression testing involves the creation of a set of tests which serve to demonstrate that the system works as expected. These would be run again many times over a testing project, when changes are made, as discussed above. This repetition of tests makes regression testing suitable for automation in many cases.

You may follow the complete series of Software Life Cycles articles here:

Software Testing Development Models
Test Levels
Software Testing Types
Maintenance Testing

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

In our previous article, we have discussed Software Testing Development Models.

For both types of development, testing plays a significant role. Testing helps to ensure that the work-products are being developed in the right way (verification) and that the product will meet the user needs (validation).

Characteristics of good testing across the development life cycle include:

• Early test design—In the V-model, we saw that test planning begins with the specification documents. This activity is part of the fundamental test process. After test planning, the documents would be analyzed and test cases designed. This approach would ensure that testing starts with the development of the requirements, i.e. a proactive approach to testing is undertaken. As we saw in iterative development, test-driven development may be adopted, pushing testing to the front of the development activity.
• Each work-product is tested—In the V-model, each document on the left is tested by an activity on the right. Each specification document is called the test basis, i.e. it is the basis on which tests are created. In iterative development, each release is tested before moving on to the next.
• Testers are involved in reviewing requirements before they are released—In the V-model, testers would be invited to review all documents from a testing perspective.

The test stages of the V-model are shown. They are often called test levels. The term test level provides an indication of the focus of the testing, and the types of problems it is likely to uncover. The typical levels of testing are:

• Unit (component) testing
• Integration testing
• System testing
• Acceptance testing

Each of these test levels will include tests designed to uncover problems specifically at that stage of development. These levels of testing can be applied to iterative development also. In addition, the levels may change depending on the system. For instance, if the system includes some software developed by external parties, or bought off the shelf, acceptance testing on these may be conducted before testing the system as a whole.

You may follow the complete series of Software Life Cycles articles here:

Software Testing Development Models
Test Levels
Software Testing Types
Maintenance Testing

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

A development life cycle for a software product involves capturing the initial requirements from the customer, expanding on these to provide the detail required for code production, writing the code and testing the product, ready for release.

A simple development model is shown below. This is known traditionally as the waterfall model.

Waterfall model

The waterfall model in above figure shows the steps in sequence where the customer requirements are progressively refined to the point where coding can take place. This type of model is often referred to as a linear or sequential model. Each work-product or activity is completed before moving on to the next.

In the waterfall model, testing is carried out once the code has been fully developed. Once this is completed, a decision can be made on whether the product can be released into the live environment.

This model for development shows how a fully tested product can be created, but it has a significant drawback: what happens if the product fails the tests? Let us look at a simple case study.

CASE STUDY—DEVELOPMENT PROCESS

In a factory environment producing rivets for an aircraft fuselage, checks are made by operators to assess the rivets on a conveyor belt. This assessment may reveal a percentage of the rivets to be defective. Usually this percentage is small, and does not result in the whole batch of rivets being rejected. Therefore the bulk of the product can be released.

Consider now the same aircraft, but the product is the software controlling the display provided for the aircrew. If, at the point of testing, too many defects are found, what happens next? Can we release just parts of the system?

In the waterfall model, the testing at the end serves as a quality check. The product can be accepted or rejected at this point. As we saw in the case of rivet production, a single point of quality checking may be acceptable, assuming that most rivets pass the quality check.

In software development, however, it is unlikely that we can simply reject the parts of the system found to be defective, and release the rest. The nature of software functionality is such that removal of software is often not a clean-cut activity—this action could well cause other areas to function incorrectly. It may even cause the system to become unusable.

In addition, we may not be able to choose not to deliver anything at all. The commercial and financial effects of this course of action could be substantial.

What is needed is a process that assures quality throughout the development life cycle. At every stage, a check should be made that the work-product for that stage meets its objectives. This is a key point, work-product evaluation taking place at the point where the product has been declared complete by its creator. If the work-product passes its evaluation (test), we can progress to the next stage in confidence. In addition, finding problems at the point of creation should make fixing any problems cheaper than fixing them at a later stage. This is the cost escalation model as covered in the post.

The checks throughout the life cycle include verification and validation.

Verification—checks that the work-product meets the requirements set out for it. An example of this would be to ensure that a website being built follows the guidelines for making websites usable by as many people as possible. Verification helps to ensure that we are building the product in the right way.

Validation—changes the focus of work-product evaluation to evaluation against user needs. This means ensuring that the behavior of the work-product matches the customer needs as defined for the project. For example, for the same website above, the guidelines may have been written with people familiar with websites in mind. It may be that this website is also intended for novice users. Validation would include these users checking that they too can use the website easily. Validation helps to ensure that we are building the right product as far as the users are concerned.

There are two types of development model that facilitate early work-product evaluation.

The first is an extension to the waterfall model, known as the V-model. The second is a cyclical model, where the coding stage often begins once the initial user needs have been captured. Cyclical models are often referred to as iterative models.

We will consider first the V-model.

V-Model (Sequential Development Model)

There are many variants of the V-model. One of these is shown in below Figure.

V-model for software development

As for the waterfall model, the left-hand side of the model focuses on elaborating the initial requirements, providing successively more technical detail as the development progresses. In the model shown, these are:

• Requirement specification—capturing of user needs.
• Functional specification—definition of functions required to meet user needs.
• Technical specification—technical design of functions identified in the functional specification.
• Program specification—detailed design of each module or unit to be built to meet required functionality.

These specifications could be reviewed to check for the following:

• Conformance to the previous work-product (so in the case of the functional specification, verification would include a check against the requirement specification).
• That there is sufficient detail for the subsequent work-product to be built correctly (again, for the functional specification, this would include a check that there is sufficient information in order to create the technical specification).
• That it is testable—is the detail provided sufficient for testing the work-product?

The middle of the V-model shows that planning for testing should start with each work-product. Thus, using the requirement specification as an example, acceptance testing would be planned for, right at the start of the development.

The right-hand side focuses on the testing activities. For each work-product, a testing activity is identified. These are shown in above Figure:

Testing against the requirement specification takes place at the acceptance testing stage.
Testing against the functional specification takes place at the system testing stage.
Testing against the technical specification takes place at the integration testing stage.
Testing against the program specification takes place at the unit testing stage.

This allows testing to be concentrated on the detail provided in each work-product, so that defects can be identified as early as possible in the life cycle, when the work-product has been created.

Remembering that each stage must be completed before the next one can be started, this approach to software development pushes validation of the system by the user representatives right to the end of the life cycle. If the customer needs were not captured accurately in the requirement specification, or if they change, then these issues may not be uncovered until the user testing is carried out.

Iterative-Incremental Development Models

Let us now look at a different model for software development—iterative development. This is one where the requirements do not need to be fully defined before coding can start. Instead, a working version of the product is built, in a series of stages, or iterations—hence the name iterative or incremental development. Each stage encompasses requirements definition, design, code and test. This is shown diagrammatically in below figure.

Iterative development

This type of development is often referred to as cyclical—we go ‘round the development cycle a number of times’, within the project. The project will have a defined timescale and cost. Within this, the cycles will be defined. Each cycle will also have a defined timescale and cost. The cycles are commonly referred to as time-boxes. For each time-box, a requirement is defined and a version of the code is produced, which will allow testing by the user representatives. At the end of each time-box, a decision is made on what extra functionality needs to be created for the next iteration. This process is then repeated until a fully working system has been produced.

A key feature of this type of development is the involvement of user representatives in the testing. Having the users represented throughout minimizes the risk of developing an unsatisfactory product. The user representatives are empowered to request changes to the software, to meet their needs.

This approach to software development can pose problems, however.

The lack of formal documentation makes it difficult to test. To counter this, developers may use test-driven development. This is where functional tests are written first, and code is then created and tested. It is reworked until it passes the tests.

In addition, the working environment may be such that developers make any changes required, without formally recording them. This approach could mean that changes cannot be traced back to the requirements or to the parts of the software that have changed. Thus, traceability as the project progresses is reduced. To mitigate this, a robust process must be put in place at the start of the project to manage these changes.

Another issue associated with changes is the amount of testing required to ensure that implementation of the changes does not cause unintended changes to other parts of the software (this is called regression testing)

Forms of iterative development include prototyping, rapid application development (RAD) and agile software development. A proprietary methodology is the Rational Unified Process (RUP).

You may follow the complete series of Software Life Cycles articles here:

Software Testing Development Models
Test Levels
Software Testing Types
Maintenance Testing

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

You may follow the complete series of Fundamentals of Testing articles here:

Why a Software Fails?
Keeping Software Test Under Control
What Testing is and What Testing Does
Software Testing Principles
Fundamental Software Test Processes
Psychology of Software Testing
Testers Code of Ethics
ISTQB Sample Questions

Que1: Which of the following is correct?
Debugging is:

a. Testing/checking whether the software performs correctly.
b. Checking that a previously reported defect has been corrected.
c. Identifying the cause of a defect, repairing the code and checking the fix is correct.
d. Checking that no unintended consequences have occurred as a result of a fix.

Answers

Ans1: The correct answer is c.
a. is a brief definition of testing.
b. is retesting.
d. is regression testing.

Que2: Which of the following are aids to good communication, and which hinder it?
i. Try to understand how the other person feels.
ii. Communicate personal feelings, concentrating upon individuals.
iii. Confirm the other person has understood what you have said and vice versa.
iv. Emphasise the common goal of better quality.
v. Each discussion is a battle to be won.

a. (i), (ii) and (iii) aid, (iv) and (v) hinder.
b. (iii), (iv) and (v) aid, (i) and (ii) hinder.
c. (i), (iii) and (iv) aid, (ii) and (v) hinder.
d. (ii), (iii) and (iv) aid, (i) and (v) hinder.

Answers

The correct answer is c.

Que3: Which option is part of the ‘implementation and execution’ area of the fundamental test process?
a. Developing the tests.
b. Comparing actual and expected results.
c. Writing a test summary.
d. Analyzing lessons learnt for future releases.

Answers

The correct answer is b.
a. is part of ‘Analysis and design’.
c. is part of ‘Evaluating exit criteria and reporting’.
d. is part of ‘Test closure activities’.

Que4:
The five parts of the fundamental test process have a broad chronological order. Which of the options gives three different parts in the correct order?
a. Implementation and execution, planning and control, analysis and design.
b. Analysis and design, evaluating exit criteria and reporting, test closure activities.
c. Evaluating exit criteria and reporting, implementation and execution, analysis and design.
Evaluating exit criteria and reporting, test closure activities, analysis and design.

Answers
The correct answer is b.
All other answers have at least one stage of the fundamental test process in the wrong sequence.

Que5:
Which pair of definitions is correct?
a. Regression testing is checking that the reported defect has been fixed; retesting is testing that there are no additional problems in previously tested software.
b. Regression testing is checking there are no additional problems in previously tested software; retesting enables developers to isolate the problem.
c. Regression testing involves running all tests that have been run before; retesting runs new tests.
d. Regression testing is checking that there are no additional problems in previously tested software, retesting is demonstrating that the reported defect has been fixed.

Answers

The correct answer is d.
Regression testing is testing that nothing has regressed. Retesting (or confirmation testing) confirms the fix is correct by running the same test after the fix has been made. No other option has both of these as true.

Que6:
Which statement is most true?
a. Different testing is needed depending upon the application.
b. All software is tested in the same way.
c. A technique that finds defects will always find defects.
d. A technique that has found no defects is not useful.

Answers

The correct answer is a.
This is a restatement of the testing principle ‘Testing is context dependent’.

Que7:
When is testing complete?
a. When time and budget are exhausted.
b. When there is enough information for sponsors to make an informed decision about release.
c. When there are no remaining high priority defects outstanding.
d. When every data combination has been exercised successfully.

Answers

The correct answer is b.
Sometimes time/money does signify the end of testing, but it is really complete when everything that was set out in advance has been achieved.

Que8:
Which list of levels of tester independence is in the correct order, starting with the most independent first?
a. Tests designed by the author; tests designed by another member of the development team; tests designed by someone from a different company.
b. Tests designed by someone from a different department within the company; tests designed by the author; tests designed by someone from a different company.
c. Tests designed by someone from a different company; tests designed by someone from a different department within the company; tests designed by another member of the development team.
d. Tests designed by someone from a different department within the company; tests designed by someone from a different company; tests designed by the author.

Answers

The correct answer is c.
This option has someone nearer to the written code in each statement. All other options are not in this order.

Que9:
The following statements relate to activities that are part of the fundamental test process.
i. Evaluating the testability of requirements.
ii. Repeating testing activities after changes.
iii. Designing the test environment set-up.
iv. Developing and prioritising test cases.
v. Verifying the environment is set up correctly.

Which statement below is TRUE?

a. (i) and (ii) are part of analysis and design, (iii), (iv) and (v) are part of test implementation and execution.
b. (i) and (iii) are part of analysis and design, (ii), (iv) and (v) are part of test implementation and execution.
c. (i) and (v) are part of analysis and design, (ii), (iii) and (iv) are part of test implementation and execution.
d. (i) and (iv) are part of analysis and design, (ii), (iii) and (v) are part of test implementation and execution.

Answers

The correct answer is b.
All other answers contain an activity identified as analysis and design that is part of implementation and test execution.

Que10:
Which statement correctly describes the public and profession aspects of the code of ethics?
a. Public: Certified software testers shall act in the best interests of their client and employer (being consistent with the wider public interest). Profession: Certified software testers shall advance the integrity and reputation of their industry consistent with the public interest.
b. Public: Certified software testers shall advance the integrity and reputation of the profession consistent with the public interest. Profession: Certified software testers shall consider the wider public interest in their actions.
c. Public: Certified software testers shall consider the wider public interest in their actions. Profession: Certified software testers shall participate in lifelong learning regarding the practice of their profession and shall promote an ethical approach to the practice of their profession.
d. Public: Certified software testers shall consider the wider public interest in their actions. Profession: Certified software testers shall advance the integrity and reputation of their industry consistent with the public interest.

Answers

The correct answer is d.
All the answers reflect the definition of two of the items from the code of ethics, and care must be taken in searching for the Public item because ‘public’ or ‘public interest’ are used in several of the eight items in the code. The key is that ‘public’ is the main item, rather than a subsidiary. In the order given in the options, a. reflects Client and employer and Profession while b. gives Profession and Public (the correct choices, but the wrong way round). Option c. gives Public and Self, leaving the last option d. to give Public and Profession.

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

In our previous article, we have discussed Psychology of Software Testing.

Testers must adhere to a code of ethics: they are required to act in a professional manner. Testers can have access to confidential and/or privileged information, and they are to treat any information with care and attention, and act responsibly to the owner(s) of this information, employers and the wider public interest. Of course, anyone can test software, so the declaration of this code of conduct applies to those who have achieved software testing certification! The code of ethics applies to the following areas:

• Public—Certified software testers shall consider the wider public interest in their actions.
• Client and employer—Certified software testers shall act in the best interests of their client and employer (being consistent with the wider public interest).
• Product—Certified software testers shall ensure that the deliverables they provide (for any products and systems they work on) meet the highest professional standards possible.
• Judgement—Certified software testers shall maintain integrity and independence in their professional judgement.
• Management—Certified software test managers and leaders shall subscribe to and promote and ethical approach to the management of software testing.
• Profession—Certified software testers shall advance the integrity and reputation of the profession consistent with the public interest.
• Colleagues—Certified software testers shall be fair to and supportive of their colleagues, and promote cooperation with software developers.
• Self—Certified software testers shall participate in lifelong learning regarding the practice of their profession, and shall promote an ethical approach to the practice of the profession.

To check your understanding, I would again like to ask you some questions:

Why do testers need to consider how they treat information they use?
What eight areas should testers consider when conducting themselves in the workplace?

You may follow the complete series of Fundamentals of Testing articles here:

Why a Software Fails?
Keeping Software Test Under Control
What Testing is and What Testing Does
Software Testing Principles
Fundamental Software Test Processes
Psychology of Software Testing
Testers Code of Ethics
ISTQB Sample Questions

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

In our previous article, we have discussed Fundamental Software Test Processes.

A variety of different people may be involved in the total testing effort, and they may be drawn from a broad set of backgrounds. Some will be developers, some professional testers, and some will be specialists, such as those with performance testing skills, whilst others may be users drafted in to assist with acceptance testing. Who ever is involved in testing needs at least some understanding of the skills and techniques of testing to make an effective contribution to the overall testing effort.

Testing can be more effective if it is not undertaken by the individual(s) who wrote the code, for the simple reason that the creator of anything (whether it is software or a work of art) has a special relationship with the created object. The nature of that relationship is such that flaws in the created object are rendered invisible to the creator. For that reason it is important that someone other than the creator should test the object. Of course we do want the developer who builds a component or system to debug it, and even to attempt to test it, but we accept that testing done by that individual cannot be assumed to be complete. Developers can test their own code, but it requires a mindset change, from that of a developer (to prove it works) to that of a tester (trying to show that it does not work). If there are separate individuals involved, there are no potential conflicts of interest. We therefore aim to have the software tested by someone who was not involved in the creation of the software; this approach is called test independence. Below are people who could test software, listed in order of increasing independence:

• Those who wrote the code.
• Members of the same development team.
• Members of a different group (independent test team).
• Members of a different company (a testing consultancy/outsourced).

Of course independence comes at a price; it is much more expensive to use a testing consultancy than to test a program oneself.

Testers and developers think in different ways. However, although we know that testers should be involved from the beginning, it is not always good to get testers involved in code execution at an early stage; there are advantages and disadvantages. Getting developers to test their own code has advantages (as soon as problems are discovered, they can be fixed, without the need for extensive error logs), but also difficulties (it is hard to find your own mistakes). People and projects have objectives, and we all modify actions to blend in with the goals.

If a developer has a goal of producing acceptable software by certain dates, then any testing is aimed towards that goal. If a defect is found in software, the software author may see this as criticism. Testers need to use tact and diplomacy when raising defect reports. Defect reports need to be raised against the software, not against the individual who made the mistake. The mistake may be in the code written, or in one of the documents upon which the code is based (requirement documents or system specification). When we raise defects in a constructive way, bad feeling can be avoided.

We all need to focus on good communication, and work on team building. Testers and developers are not opposed, but working together, with the joint target of better quality systems. Communication needs to be objective, and expressed in impersonal ways:

• The aim is to work together rather than be confrontational. Keep the focus on delivering a quality product.
• Results should be presented in a non-personal way. The work-product may be wrong, so say this in a non-personal way.
• Attempt to understand how others feel; it is possible to discuss problems and still leave all parties feeling positive.
• At the end of discussions, confirm that you have both understood and been understood. ‘So, am I right in saying that your aim was to deliver on Friday by 12:00, even if you knew there were problems?’

As testers and developers, one of our goals is better quality systems delivered in a timely manner. Good communication between testers and the development teams is one way that this goal can be reached.

To check your understanding, I would again like to ask you some questions:

When testing software, who has the highest level of independence?
Contrast the advantages and disadvantages of developers testing their own code.
Suggest three ways that confrontation can be avoided.

You may follow the complete series of Fundamentals of Testing articles here:

Why a Software Fails?
Keeping Software Test Under Control
What Testing is and What Testing Does
Software Testing Principles
Fundamental Software Test Processes
Psychology of Software Testing
Testers Code of Ethics
ISTQB Sample Questions

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

In our previous article, we have discussed General Software Testing Principles?

The most visible part of testing is running one or more tests: test execution. We also have to prepare for running tests, analyze the tests that have been run, and see whether testing is complete. Both planning and analyzing are very necessary activities that enhance and amplify the benefits of the test execution itself. It is no good testing without deciding how, when and what to test. Planning is also required for the less formal test approaches such as exploratory testing.

The fundamental test process consists of five parts that encompass all aspects of testing (See below Figure):

• Planning and control.
• Analysis and design.
• Implementation and execution.
• Evaluating exit criteria and reporting.
• Test closure activities.

Fundamental test process

Although the main activities are in a broad sequence, they are not undertaken in a rigid way. An earlier activity may need to be revisited. A defect found in test execution can sometimes be resolved by adding functionality that was originally not present (either missing in error, or the new facilities are needed to make the other part correct). The new features themselves have to be tested, so even though implementation and execution are in progress, the ‘earlier’ activity of analysis and design has to be performed for the new features.

Iteration of activities

We sometimes need to do two or more of the main activities in parallel. Time pressure can mean that we begin test execution before all tests have been designed.

Test Planning and Control

Planning is determining what is going to be tested, and how this will be achieved. It is where we draw a map; how activities will be done; and who will do them. Test planning is also where we define the test completion criteria. Completion criteria are how we know when testing is finished. Control, on the other hand, is what we do when the activities do not match up with the plans. It is the ongoing activity where we compare the progress against the plan. As progress takes place, we may need to adjust plans to meet the targets, if this is possible. Therefore we need to undertake both planning and control throughout the testing activities. We plan at the outset, but as testing progresses, undertake monitoring and control activities (monitoring to measure what has happened, control to adjust future activities in the light of experience). Monitoring and control feed back into the continual activity of planning.

Test Analysis and Design

Analysis and design are concerned with the fine detail of what to test (test conditions), and how to combine test conditions into test cases, so that a small number of test cases can cover as many of the test conditions as possible. The analysis and design stage is the bridge between planning and test execution. It is looking backward to the planning (schedules, people, what is going to be tested) and forward to the execution activity (test expected results, what environment will be needed).

A part of the design process needs to consider the test data that will be required for the test conditions and test cases that have been drawn up.

Test design involves predicting how the software under test should behave in a given set of circumstances. Sometimes the expected outcome of a test is trivial: when ordering books from an online book retailer, for instance, under no circumstances should money be refunded to the customer's card without intervention from a supervisor. If we do not detail expected outcomes before starting test execution, there is a real danger that we will miss the one item of detail that is vital, but wrong.

The main points of this activity are as follows:

• Reviewing requirements, architecture, design, interface specifications and other parts, which collectively comprise the test basis.
• Analyzing test items, the specification, behavior and structure to identify test conditions and test data required.
• Designing the tests, including assigning priority.
• Determining whether the requirements and the system are testable.
• Detailing what the test environment should look like, and whether there are any infrastructure and tools required.
• Highlighting the test data required for the test conditions and test cases.
• Creating bi-directional traceability between test basis and test cases.

Test Implementation and Execution

The test implementation and execution activity involves running tests, and this will include where necessary any set-up/tear-down activities for the testing. It will also involve checking the test environment before testing begins. Test execution is the most visible part of testing, but it is not possible without other parts of the fundamental test process. It is not just about running tests. As we have already mentioned, the most important tests need to be run first. How do we know what are the most important tests to run? This is determined during the planning stages, and refined as part of test design.

One important aspect undertaken at this stage is combining test cases into an overall run procedure, so that test time can be utilized efficiently. Here the logical ordering of tests is important so that, where possible, the outcome of one test creates the preconditions for one or more tests that are later in the execution sequence.

As tests are run, their outcome needs to be logged, and a comparison made between expected results and actual results. Whenever there is a discrepancy between the expected and actual results, this needs to be investigated. If necessary a test incident should be raised. Each incident requires investigation, although corrective action will not be necessary in every case.

When anything changes (software, data, installation procedures, user documentation, etc.), we need to do two kinds of testing on the software. First of all, tests should be run to make sure that the problem has been fixed. We also need to make sure that the changes have not broken the software elsewhere. These two types are usually called retesting and regression testing, respectively. In retesting we are looking in fine detail at the changed area of functionality, whereas regression testing should cover all the main functions to ensure that no unintended changes have occurred. On a financial system, we should include end of day/end of month/end of year processing, for example, in a regression test pack.

Test implementation and execution is where the most visible test activities are undertaken, and usually have the following parts:

• Developing and prioritizing test cases, creating test data, writing test procedures and, optionally, preparing test harnesses and writing automated test scripts.
• Collecting test cases into test suites, where tests can be run one after another for efficiency.
• Checking the test environment set-up is correct.
• Running test cases in the determined order. This can be manually or using test execution tools.
• Keeping a log of testing activities, including the outcome (pass/fail) and the versions of software, data, tools and test ware (scripts, etc.).
• Comparing actual results with expected results.
• Reporting discrepancies as incidents with as much information as possible, including if possible causal analysis (code defect, incorrect test specification, test data error or test execution error).
• Where necessary, repeating test activities when changes have been made following incidents raised. This includes re-execution of a test that previously failed in order to confirm a fix (retesting), execution of a corrected test and execution of previously passed tests to check that defects have not been introduced (regression testing).

Evaluating Exit Criteria and Reporting

Remember that exit criteria were defined during test planning and before test execution started. At the end of test execution, the test manager checks to see if these have been met. If the criterion was that there would be 85 per cent statement coverage (i.e. 85 per cent of all executable statements have been executed, and as a result of execution the figure is 75 per cent, there are two possible actions: change the exit criteria, or run more tests. It is possible that even if the preset criteria were met, more tests would be required. Also, writing a test summary for stakeholders would say what was planned, what was achieved, highlight any differences and in particular things that were not tested.

The fourth stage of the fundamental test process, evaluating exit criteria, comprises the following:

• Checking whether the previously determined exit criteria have been met.
• Determining if more tests are needed or if the specified exit criteria need amending.
• Writing up the result of the testing activities for the business sponsors and other stakeholders.

Test Closure Activities

Testing at this stage has finished. Test closure activities concentrate on making sure that everything is tidied away, reports written, defects closed, and those defects deferred for another phase clearly seen to be as such.
At the end of testing, the test closure stage is composed of the following:

• Ensuring that the documentation is in order; what has been delivered is defined (it may be more or less than originally planned), closing incidents and raising changes for future deliveries, documenting that the system has been accepted.
• Closing down and archiving the test environment, test infrastructure and testware used.
• Passing over testware to the maintenance team.
• Writing down the lessons learned from this testing project for the future, and incorporating lessons to improve the testing process (‘testing maturity’).

To check your understanding, I would again like to ask you some questions:

What are the stages in the fundamental test process (in the correct sequence)?
Briefly compare regression testing and retesting.
When should the expected outcome of a test be defined?

You may follow the complete series of Fundamentals of Testing articles here:

Why a Software Fails?
Keeping Software Test Under Control
What Testing is and What Testing Does
Software Testing Principles
Fundamental Software Test Processes
Psychology of Software Testing
Testers Code of Ethics
ISTQB Sample Questions

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

In our previous article, we have discussed What Testing is and what Testing Does?

Testing is a very complex activity, and the software problems described earlier highlight that it can be difficult to do well. We now describe some general testing principles that help testers, principles that have been developed over the years from a variety of sources. These are not all obvious, but their purpose is to guide testers, and prevent the types of problems described previously.

Testing Shows the Presence of Bugs

Running a test through a software system can only show that one or more defects exist. Testing cannot show that the software is error free. Consider whether the top 10 wanted criminals website was error free. There were no functional defects, yet the website failed. In this case the problem was non-functional and the absence of defects was not adequate as a criterion for release of the website into operation.

In our coming posts, we shall discuss retesting, when a previously failed test is rerun, to show that under the same conditions, the reported problem no longer exists. In this type of situation, testing can show that one particular problem no longer exists.

Although there may be other objectives, usually the main purpose of testing is to find defects. Therefore tests should be designed to find as many defects as possible.

Exhaustive Testing is Impossible

If testing finds problems, then surely you would expect more testing to find additional problems, until eventually we would have found them all. We discussed exhaustive testing earlier when looking at the Ariane 5 rocket launch, and concluded that for large complex systems, exhaustive testing is not possible. However, could it be possible to test small pieces of software exhaustively, and only incorporate exhaustively tested code into large systems?

Exhaustive testing – a test approach in which all possible data combinations are used. This includes implicit data combinations present in the state of the software/data at the start of testing.

Consider a small piece of software where one can enter a password, specified to contain up to three characters, with no consecutive repeating entries. Using only western alphabetic capital letters and completing all three characters, there are 26 × 26 × 26 input permutations (not all of which will be valid). However, with a standard keyboard, there are not 26 × 26 × 26 permutations, but a much higher number, 256 × 256 × 256, or 224. Even then, the number of possibilities is higher. What happens if three characters are entered, and the ‘delete last character’ right arrow key removes the last two? Are special key combinations accepted, or do they cause system actions (Ctrl + P, for example)? What about entering a character, and waiting 20 minutes before entering the other two characters? It may be the same combination of keystrokes, but the circumstances are different. We can also include the situation where the 20-minute break occurs over the change-of-day interval. It is not possible to say whether there are any defects until all possible input combinations have been tried.

Even in this small example, there are many, many possible data combinations to attempt.

Unless the application under test (AUT) has an extremely simple logical structure and limited input, it is not possible to test all possible combinations of data input and circumstances. For this reason, risk and priorities are used to concentrate on the most important aspects to test. Both ‘risk’ and ‘priorities’ are covered later in more detail. Their use is important to ensure that the most important parts are tested.

Early Testing

When discussing why software fails, we briefly mentioned the idea of early testing. This principle is important because, as a proposed deployment date approaches, time pressure can increase dramatically. There is a real danger that testing will be squeezed, and this is bad news if the only testing we are doing is after all the development has been completed. The earlier the testing activity is started, the longer the elapsed time available. Testers do not have to wait until software is available to test.

Work-products are created throughout the software development life cycle (SDLC). As soon as these are ready, we can test them. Requirement documents are the basis for acceptance testing, so the creation of acceptance tests can begin as soon as requirement documents are available. As we create these tests, it will highlight the contents of the requirements. Are individual requirements testable? Can we find ambiguous or missing requirements?

Many problems in software systems can be traced back to missing or incorrect requirements. The use of reviews can break the ‘error—defect—failure’ cycle. In early testing we are trying to find errors and defects before they are passed to the next stage of the development process. Early testing techniques are attempting to show that what is produced as a system specification, for example, accurately reflects that which is in the requirement documents. Ed Kit (Kit, 1995) discusses identifying and eliminating errors at the part of the SDLC in which they are introduced. If an error/defect is introduced in the coding activity, it is preferable to detect and correct it at this stage. If a problem is not corrected at the stage in which it is introduced, this leads to what Kit calls ‘errors of migration’. The result is rework. We need to rework not just the part where the mistake was made, but each subsequent part where the error has been replicated. A defect found at acceptance testing where the original mistake was in the requirements will require several work-products to be reworked, and subsequently to be retested.

Comparative cost to correct errors

Stage error is found	Comparative cost
Requirements	$1
Coding	$10
Program testing	$100
System testing	$1,000
User acceptance testing	$10,000
Live running	$100,000

This is known as the cost escalation model.

What is undoubtedly true is that the graph of the relative cost of early and late identification/correction of defects rises very steeply as shown in below figure.

Defect Clustering

Problems do occur in software! It is a fact. Once testing has identified (most of) the defects in a particular application, it is at first surprising that the spread of defects is not uniform. In a large application, it is often a small number of modules that exhibit the majority of the problems. This can be for a variety of reasons, some of which are:

• System complexity.
• Volatile code.
• The effects of change upon change.
• Development staff experience.
• Development staff inexperience.

This is the application of the Pareto principle to software testing: approximately 80 per cent of the problems are found in about 20 per cent of the modules. It is useful if testing activity reflects this spread of defects, and targets areas of the application under test where a high proportion of defects can be found. However, it must be remembered that testing should not concentrate exclusively on these parts. There may be fewer defects in the remaining code, but testers still need to search diligently for them.

The Pesticide Paradox

Running the same set of tests continually will not continue to find new defects. Developers will soon know that the test team always tests the boundaries of conditions, for example, so they will test these conditions before the software is delivered. This does not make defects elsewhere in the code less likely, so continuing to use the same test set will result in decreasing effectiveness of the tests. Using other techniques will find different defects.

For example, a small change to software could be specifically tested and an additional set of tests performed, aimed at showing that no additional problems have been introduced (this is known as regression testing). However, the software may fail in production because the regression tests are no longer relevant to the requirements of the system or the test objectives. Any regression test set needs to change to reflect business needs, and what are now seen as the most important risks.

Testing is Context Dependent

Different testing is necessary in different circumstances. A website where information can merely be viewed will be tested in a different way to an e-commerce site, where goods can be bought using credit/debit cards. We need to test an air traffic control system with more rigour than an application for calculating the length of a mortgage.

Risk can be a large factor in determining the type of testing that is needed. The higher the possibility of losses, the more we need to invest in testing the software before it is implemented.

For an e-commerce site, we should concentrate on security aspects. Is it possible to bypass the use of passwords? Can ‘payment’ be made with an invalid credit card, by entering excessive data into the card number? Security testing is an example of a specialist area, not appropriate for all applications. Such types of testing may require specialist staff and software tools.

Absence of Errors Fallacy

Software with no known errors is not necessarily ready to be shipped. Does the application under test match up to the users' expectations of it? The fact that no defects are outstanding is not a good reason to ship the software.

Before dynamic testing has begun, there are no defects reported against the code delivered. Does this mean that software that has not been tested (but has no outstanding defects against it) can be shipped? We think not!

To check your understanding, I would again like to ask you some questions:

Why is ‘zero defects’ an insufficient guide to software quality?
Give three reasons why defect clustering may exist.
Briefly justify the idea of early testing.

You may follow the complete series of Fundamentals of Testing articles here:

Why a Software Fails?
Keeping Software Test Under Control
What Testing is and What Testing Does
Software Testing Principles
Fundamental Software Test Processes
Psychology of Software Testing
Testers Code of Ethics
ISTQB Sample Questions

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

In our previous article, we have discussed Keeping Software Under Control

So far we have worked with an intuitive idea of what testing is. We have recognized that it is an activity used to reduce risk and improve quality by finding defects, which is all true. However, we need to understand a little more about how software testing works in practice before we can think about how to implement effective testing.

Testing and Debugging

Testing and debugging are different kinds of activity, both of which are very important. Debugging is the process that developers go through to identify the cause of bugs or defects in code and undertake corrections. Ideally some check of the correction is made, but this may not extend to checking that other areas of the system have not been inadvertently affected by the correction. Testing, on the other hand, is a systematic exploration of a component or system with the main aim of finding and reporting defects. Testing does not include correction of defects—these are passed on to the developer to correct. Testing does, however, ensure that changes and corrections are checked for their effect on other parts of the component or system.

Effective debugging is essential before testing begins to raise the level of quality of the component or system to a level that is worth testing, i.e. a level that is sufficiently robust to enable rigorous testing to be performed. Debugging does not give confidence that the component or system meets its requirements completely. Testing makes a rigorous examination of the behavior of a component or system and reports all defects found for the development team to correct. Testing then repeats enough tests to ensure that defect corrections have been effective. So both are needed to achieve a quality result.

Static Testing and Dynamic Testing

Static testing is the term used for testing where the code is not exercised. This may sound strange, but remember that failures often begin with a human error, namely a mistake in a document such as a specification. We need to test these because errors are much cheaper to fix than defects or failures (as you will see). That is why testing should start as early as possible, another basic principle explained in more detail later in this chapter. Static testing involves techniques such as reviews, which can be effective in preventing defects, e.g. by removing ambiguities and errors from specification documents; this is a topic in its own right. Dynamic testing is the kind that exercises the program under test with some test data, so we speak of test execution in this context. The discipline of software testing encompasses both static and dynamic testing.

Testing as a Process

We have already seen that there is much more to testing than test execution. Before test execution there is some preparatory work to do to design the tests and set them up; after test execution there is some work needed to record the results and check whether the tests are complete. Even more important than this is deciding what we are trying to achieve with the testing and setting clear objectives for each test. A test designed to give confidence that a program functions according to its specification, for example, will be quite different from one designed to find as many defects as possible. We define a test process to ensure that we do not miss critical steps and that we do things in the right order.

Testing as a Set of Techniques

The final challenge is to ensure that the testing we do is effective testing. It might seem paradoxical, but a good test is one that finds a defect if there is one present. A test that finds no defect has consumed resources but added no value; a test that finds a defect has created an opportunity to improve the quality of the product. How do we design tests that find defects? We actually do two things to maximize the effectiveness of the tests. techniques are all based on certain testing principles that have been discovered and documented over the years, and these principles are the second mechanism we use to ensure that tests are effective. Even when we cannot apply rigorous test design for some reason (such as time pressures) we can still apply the general principles to guide our testing.

To check your understanding, I would again like to ask you some questions:

Describe static testing and dynamic testing.
What is debugging?
What other elements apart from ‘test execution’ are included in ‘testing’?

You may follow the complete series of Fundamentals of Testing articles here:

Why a Software Fails?
Keeping Software Test Under Control
What Testing is and What Testing Does
Software Testing Principles
Fundamental Software Test Processes
Psychology of Software Testing
Testers Code of Ethics
ISTQB Sample Questions

To see all articles of ISTQB-ISEB Foundation guide, see here:

Software Testing-ISTQB ISEB Foundation Guide

In our previous article, we have discussed Why a Software Fails?

With all of the examples we have seen so far in our previous posts, what common themes can we identify? There may be several themes that we could draw out of the examples, but one theme is clear: either insufficient testing or the wrong type of testing was done. More and better software testing seems a reasonable aim, but that aim is not quite as simple to achieve as we might expect.

Exhaustive Testing of Complex Systems is Not Possible

With the Ariane 5 rocket launch, a particular software module was reused from the Ariane 4 program. Only part of the functionality of the module was required, but the module was incorporated without changes. The unused functionality of the reused module indirectly caused a directional nozzle to move in an uncontrolled way because certain variables were incorrectly updated. In an Ariane 4 rocket the module would have performed as required, but in the Ariane 5 environment this malfunction in an area of software not even in use caused a catastrophic failure. The failure is well documented, but what is clear is that conditions were encountered in the first few seconds after the launch that were not expected, and therefore had not been tested.

If every possible test had been run, the problem would have been detected. However, if every test had been run, the testing would still be running now, and the ill-fated launch would never have taken place; this illustrates one of the general principles of software testing, which are explained below. With large and complex systems it will never be possible to test everything exhaustively; in fact it is impossible to test even moderately complex systems exhaustively.

In the Ariane 5 case it would be unhelpful to say that not enough testing was done; for this particular project, and for many others of similar complexity, that would certainly always be the case. In the Ariane 5 case the problem was that the right sort of testing was not done because the problem had not been detected.

Testing and Risk

Risk is inherent in all software development. The system may not work or the project to build it may not be completed on time, for example. These uncertainties become more significant as the system complexity and the implications of failure increase. Intuitively, we would expect to test an automatic flight control system more than we would test a video game system. Why? Because the risk is greater. There is a greater probability of failure in the more complex system and the impact of failure is also greater. What we test, and how much we test it, must be related in some way to the risk. Greater risk implies more and better testing.

Testing and Quality

Quality is notoriously hard to define. If a system meets its users' requirements that constitutes a good starting point. In the examples we looked at earlier the online tax returns system had an obvious functional weakness in allowing one user to view another user's details. While the user community for such a system is potentially large and disparate, it is hard to imagine any user that would find that situation anything other than unacceptable. In the top 10 criminals example the problem was slightly different. There was no failure of functionality in this case; the system was simply swamped by requests for access. This is an example of a non-functional failure, in that the system was not able to deliver its services to its users because it was not designed to handle the peak load that materialized after radio and TV coverage.

Of course the software development process, like any other, must balance competing demands for resources. If we need to deliver a system faster (i.e. in less time), for example, it will usually cost more. The items at the corners (or vertices) of the triangle of resources in the below figure are time, money and quality. These three affect one another, and also influence the features that are or are not included in the delivered software.

Resources Triangle

One role for testing is to ensure that key functional and non-functional requirements are examined before the system enters service and any defects are reported to the development team for rectification. Testing cannot directly remove defects, nor can it directly enhance quality. By reporting defects it makes their removal possible and so contributes to the enhanced quality of the system. In addition, the systematic coverage of a software product in testing allows at least some aspects of the quality of the software to be measured. Testing is one component in the overall quality assurance activity that seeks to ensure that systems enter service without defects that can lead to serious failures.

Deciding When ‘Enough is Enough’

How much testing is enough, and how do we decide when to stop testing?

We have so far decided that we cannot test everything, even if we would wish to. We also know that every system is subject to risk of one kind or another and that there is a level of quality that is acceptable for a given system. These are the factors we will use to decide how much testing to do.

The most important aspect of achieving an acceptable result from a finite and limited amount of testing is prioritization. Do the most important tests first so that at any time you can be certain that the tests that have been done are more important than the ones still to be done. Even if the testing activity is cut in half it will still be true that the most important testing has been done. The most important tests will be those that test the most important aspects of the system: they will test the most important functions as defined by the users or sponsors of the system, and the most important non-functional behavior, and they will address the most significant risks.
The next most important aspect is setting criteria that will give you an objective test of whether it is safe to stop testing, so that time and all the other pressures do not confuse the outcome. These criteria, usually known as completion criteria, set the standards for the testing activity by defining areas such as how much of the software is to be tested.

Priorities and completion criteria provide a basis for planning (which will be covered in above figure applies. In the end, the desired level of quality and risk may have to be compromised, but our approach ensures that we can still determine how much testing is required to achieve the agreed levels and we can still be certain that any reduction in the time or effort available for testing will not affect the balance—the most important tests will still be those that have already been done whenever we stop.

To check your level of understanding, I would like to ask few questions now.

1. Describe the interaction between errors, defects and failures.
2. Software failures can cause losses. Give three consequences of software failures.
3. What are the vertices of the ‘triangle of resources’?

You may follow the complete series of Fundamentals of Testing articles here:

Why a Software Fails?
Keeping Software Test Under Control
What Testing is and What Testing Does
Software Testing Principles
Fundamental Software Test Processes
Psychology of Software Testing
Testers Code of Ethics
ISTQB Sample Questions

In our Previous article, we have discussed Fundamentals of Software Testing

Examples of software failure are depressingly common. Here are some you may recognize:

• The first launch of the European Space Agency Ariane 5 rocket in June 1996 failed after 37½ seconds. A software error caused the rocket to deviate from its vertical ascent, and the self-destruct capabilities were enacted before the then unpredictable flight path resulted in a bigger problem.
• When the UK Government introduced online filing of tax returns, a user could sometimes see the amount that a previous user earned. This was regardless of the physical location of the two applicants.
• In November 2005, information on the UK's top 10 wanted criminals was displayed on a website. The publication of this information was described in newspapers and on morning radio and television and, as a result, many people attempted to access the site. The performance of the website proved inadequate under this load and the website had to be taken offline. The publicity created performance peaks beyond the capacity of the website.
• When a well-known online book retailer first went live, ordering a negative number of books meant that the transaction sum involved was refunded to the ‘purchaser’. Development staff had not anticipated that anyone would attempt to purchase a negative number of books. Code was developed to allow refunds to customers to be made by administrative staff—but self-requested refunds are not valid.
• A small, one-line, change in the billing system of an electrical provider blacked out the whole of a major US city.

What is it about these examples that make them so startling? Is it a sense that something fairly obvious was missed? Is it the feeling that, expensive and important as they were, the systems were allowed to enter service before they were ready? Do you think these systems were adequately tested? Obviously they were not, but in this book we want to explore why this was the case and why these kinds of failure continue to plague us.

To understand what is going on we need to start at the beginning, with the people who design systems. Do they make mistakes? Of course they do. People make mistakes because they are fallible, but there are also many pressures that make mistakes more likely. Pressures such as deadlines, complexity of systems and organizations, and changing technology all bear down on designers of systems and increase the likelihood of errors in specifications, in designs and in software code. These errors are where major system failures usually begin. If a document with an error in it is used to specify a component the component will be faulty and will probably exhibit incorrect behavior. If this faulty component is built into a system the system may fail. While failure is not always guaranteed, it is likely that errors in specifications will lead to faulty components and faulty components will cause system failure.

An error (or mistake) leads to a defect, which can cause an observed failure.

Effect of an error

There are other reasons why systems fail. Environmental conditions such as the presence of radiation, magnetism, electronic fields or pollution can affect the operation of hardware and firmware and lead to system failure.

If we want to avoid failure we must either avoid errors and faults or find them and rectify them. Testing can contribute to both avoidance and rectification, as we will see when we have looked at the testing process in a little more detail. One thing is clear: if we wish to influence errors with testing we need to begin testing as soon as we begin making errors—right at the beginning of the development process—and we need to continue testing until we are confident that there will be no serious system failures—right at the end of the development process.

Before we move on, let us just remind ourselves of the importance of what we are considering. Incorrect software can harm:

• people (e.g. by causing an aircraft crash in which people die, or by causing a hospital life support system to fail);
• companies (e.g. by causing incorrect billing, which results in the company losing money);
• the environment (e.g. by releasing chemicals or radiation into the atmosphere).

Software failures can sometimes cause all three of these at once. The scenario of a train carrying nuclear waste being involved in a crash has been explored to help build public confidence in the safety of transporting nuclear waste by train. A failure of the train's on-board systems or of the signalling system that controls the train's movements could lead to catastrophic results. This may not be likely (we hope it is not) but it is a possibility that could be linked with software failure. Software failures, then, can lead to:

• Loss of money
• Loss of time
• Loss of business reputation
• Injury
• Death

You may follow the complete series of Fundamentals of Testing articles here:

Why a Software Fails?
Keeping Software Test Under Control
What Testing is and What Testing Does
Software Testing Principles
Fundamental Software Test Processes
Psychology of Software Testing
Testers Code of Ethics
ISTQB Sample Questions