To see all articles of ISTQB-ISEB Foundation guide, see here:
Software Testing-ISTQB ISEB Foundation Guide
The specification of test cases is the second step in the fundamental test process (FTP) defined in the Introduction. The terms specification and design are used interchangeably in this context; in this section we discuss the creation of test cases by design.
The design of tests comprises three main steps:
- Identify test conditions.
- Specify test cases.
- Specify test procedures.
A test condition—an item or event of a component or system that could be verified by one or more test cases, e.g. a function, transaction, feature, quality attribute, or structural element.
In other words, a test condition is some characteristic of our software that we can check with a test or a set of tests.
A test case—a set of input values, execution preconditions, expected results and execution post conditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement.
In other words, a test case: gets the system to some starting point for the test (execution preconditions); then applies a set of input values that should achieve a given outcome (expected result), and leaves the system at some end point (execution post condition).
Our test design activity will generate the set of input values and we will predict the expected outcome by, for example, identifying from the specification what should happen when those input values are applied.
We have to define what state the system is in when we start so that it is ready to receive the inputs and we have to decide what state it is in after the test so that we can check that it ends up in the right place.
A test procedure specification—a sequence of actions for the execution of a test.
A test procedure therefore identifies all the necessary actions in sequence to execute a test. Test procedure specifications are often called test scripts (or sometimes manual test scripts to distinguish them from the automated scripts that control test execution tools).
So, going back to our three step process above, we:
- decide on a test condition, which would typically be a small section of the specification for our software under test;
- design a test case that will verify the test condition;
- write a test procedure to execute the test, i.e. get it into the right starting state, input the values, and check the outcome.
Bear in mind as well that the test development process may be implemented in more or less formal ways. In some situations it may be appropriate to produce very little documentation and in others a very formal and documented process may be appropriate. It all depends on the context of the testing, taking account of factors such as maturity of development and test processes, the amount of time available and the nature of the system under test. Safety-critical systems, for example, will normally require a formal test process.
The best way to clarify the process is to work through a simple example.
TEST CASE DESIGN BASICS
Suppose we have a system that contains the following specification for an input screen:
1.2.3 The input screen shall have three fields: a title field with a drop-down selector; a surname field that can accept up to 20 alphabetic characters and the hyphen (-) character; a first name field which can accept up to 20 alphabetic characters. All alphabetic characters shall be case insensitive. All fields must be completed. The data is validated when the Enter key is pressed. If the data is valid the system moves on to the job input screen; if not, an error message is displayed.
This specification enables us to define test conditions; for example, we could define a test condition for the surname field (i.e. it can accept up to 20 alphabetic characters and the hyphen (-) character) and define a set of test cases to test that field.
To test the surname field we would have to navigate the system to the appropriate input screen, select a title, tab to the surname field (all this would be setting the test precondition), enter a value (the first part of the set of input values), tab to the first name field and enter a value (the second part of the set of input values that we need because all fields must be completed), then press the Enter key. The system should either move on to the job input screen (if the data we input was valid) or display an error message (if the input data was not valid). Of course, we would need to test both of these cases.
The preceding paragraph is effectively the test procedure, though we might lay it out differently for real testing.
A good test case needs some extra information. First, it should be traceable back to the test condition and the element of the specification that it is testing; we can do this by applying the specification reference to the test, e.g. by identifying this test as T1.2.3.1 (because it is the first test associated with specification element 1.2.3). Secondly, we would need to add a specific value for the input, say ‘Hambling’ and ‘Brian’. Finally we would specify that the system should move to the job input screen when ‘Enter’ is pressed.
TEST CASE DESIGN EXAMPLE
As an example, we could key in the following test cases:
Mr Hambling Brian
Ms Samaroo Angelina
Ms Simmonite Compo
Mr Hyde-White Wilfred
All these would be valid test cases; even though Compo Simmonite was an imaginary male character in a TV series, the input is correct according to the specification.
We should also test some invalid inputs, such as:
Mr Thompson1 Geoff
Mr "Morgan" Peter
Mr Williams ‘Pete’
There are many more possibilities that infringe the rules in the specification, but these should serve to illustrate the point. You may be thinking that this simple specification could generate a very large number of test cases—and you would be absolutely right. One of our aims in using systematic test case design techniques will be to cut down the number of tests we need to run to achieve a given level of confidence in the software we are testing.
The test procedure would need to add some details along the following lines:
- Select the
option from the main menu. - Select the ‘input’ option from the
menu. - Select ‘Mr’ from the ‘Title’ drop-down menu.
- Check that the cursor moves to the ‘surname’ field.
- Type in ‘Hambling’ and press the tab key once; check that the cursor moves to the ‘first name’ field.
- Type in ‘Brian’ and press the Enter key.
- Check that the Job Input screen is displayed.
- …
The test procedure would collect together all the test cases related to this specification element so that they can all be executed together as a block; there would be several to test valid and non-valid inputs, as you have seen in the example.
In the wider process (the FTP) we would move on to the test execution step next. In preparation for execution the test execution schedule collects together all the tests and sequences them, taking into account any priorities (highest priority tests would be run first) and any dependencies between tests. For example, it would make sense to do all the tests on the input screen together and to do all the tests that use input data afterwards; that way we get the input screen tests to do the data entry that we will need for the later tests. There might also be technical reasons why we run tests in a particular sequence; for example, a test of the password security needs to be done at the beginning of a sequence of tests because we need to be able to get into the system to run the other tests.
You may follow the complete series of Test Design Techniques articles here:
Test Development Process
The Idea of Test Coverage
Categories of Design Techniques
Specification Black Box Techniques
Structure based Whitebox techniques
Experience based Testing
Choosing Test Techniques
To see all articles of ISTQB-ISEB Foundation guide, see here:
Software Testing-ISTQB ISEB Foundation Guide
Post a Comment