To see all articles of ISTQB-ISEB Foundation guide, see here:
Software Testing-ISTQB ISEB Foundation Guide
Test planning is the most important activity undertaken by a test leader in any test project. It ensures that there is initially a list of tasks and milestones in a baseline plan to track progress against, as well as defining the shape and size of the test effort. Test planning is used in development and implementation projects (sometimes called ‘greenfield’) as well as maintenance (change and fix) activities.
The main document produced in test planning is often called a master test plan or a project test plan. This document defines the high level of the test activities being planned. It is normally produced during the early phases of the project (e.g. initiation) and will provide sufficient information to enable a test project to be established (bearing in mind that at this point in a project little more than requirements may be available from which to plan).
The details of the test-level activities are documented within test-level plans, e.g. the system test plan. These documents will contain the detailed activities and estimates for the relevant test level.
Below figure shows where test-level test plans fit into the V-model. It shows how a test plan exists for each test level and that they will usually refer to the master test plan.
Test plans in the V-model
The contents sections of a test plan for either the master test plan or test-level plans are normally identical or very similar. IEEE 829, the Standard for Software Test Documentation, contains details of what the content of the plans should be.
Test planning is a continual activity that spans the life of the test project; it takes place in all life-cycle stages. As risks and changes occur, the plan and planning should be amended to recognize these and reflect the current position. As the plans will have been baselined (locked down) after initial sign-off, these changes would normally be managed by the project change process. Baselining a document effectively secures it from further change unless authorized via a change control process.
A useful revision aid to help remember the 16 sections of the IEEE 829 test plan is the acronym ‘SPACEDIRT’, each letter mapping to one or several sections of the test plan:
S scope (including test items, features to be tested and features not to be tested)
P people (including responsibilities, staff and training and approvals)
A approach
C criteria (including item pass/fail criteria and suspension and resumption requirements)
E environment needs
D deliverables (test)
I identifier and introduction (test plan)
R risks and contingencies
T testing tasks and schedule
Test-Planning Activities
During test planning various activities for an entire system or a part of a system have to be undertaken by those working on the plan. They include:
2. development—the building of the code;
3. implementation—the activities surrounding implementation into a live environment.
Entry criteria are used to determine when a given test activity can start. This could include the beginning of a level of testing, when test design and/or when test execution is ready to start.
Examples of some typical entry criteria to test execution (for example) may include:
Exit criteria are used to determine when a given test activity has been completed or when it should stop. Exit criteria can be defined for all of the test activities, such as planning, specification and execution as a whole, or to a specific test level for test specification as well as execution.
Exit criteria should be included in the relevant test plans.
Some typical exit criteria might be:
Exit criteria should have been agreed as early as possible in the life cycle; however, they can be and often are subject to controlled change as the detail of the project becomes better understood and therefore the ability to meet the criteria is better understood by those responsible for delivery.
Test Estimation
The syllabus details two test estimation approaches, metrics-based and expert based. The two approaches are quite different, the former being based upon data whilst the latter is a somewhat subjective approach.
The Metrics-Based Approach
This approach relies upon data collected from previous or similar projects. This kind of data might include:
It is important that the actual costs and time for testing are accurately recorded. These can then be used to re validate and possibly update the metrics for use on the next similar project.
The Expert-Based Approach
This alternative approach to metrics is to use the experience of owners of the relevant tasks or experts to derive an estimate (this is also known as the Wide Band Delphi approach). In this context ‘experts’ could be:
Many things affect the level of effort required to fulfill the test requirements of a project. These can be split into three main categories, as shown below.
2. complexity of the final product;
3. the amount of non-functional requirements;
4. the security requirements (perhaps meeting BS 7799, the security standard);
5. how much documentation is required (e.g. some legislation-driven changes demand a certain level of documentation that may be more than an organization would normally produce);
the availability and quality of the test basis (e.g. requirements and specifications).
2. amount of budget available;
3. skills of those involved in the testing and development activity (the lower the skill level in development, the more defects could be introduced, and the lower the skill level in testing, the more detailed the test documentation needs to be);
4. which tools are being used across the life cycle (i.e. the amount of automated testing will affect the effort required).
2. test cases to be written.
Taking all of this into account, once the estimate is developed and agreed the test leader can set about identifying the required resources and building the detailed plan.
You may follow the complete series of Test Management articles here:
Test Management Risk And Testing
Software Test Organization
Software Test Approaches Test Strategies
Software Test Planning And Estimation
Software Test Progress Monitoring & Control
Software Testing Incident Management
Software Test Configuration Management
To see all articles of ISTQB-ISEB Foundation guide, see here:
Software Testing-ISTQB ISEB Foundation Guide
Test planning is a continual activity that spans the life of the test project; it takes place in all life-cycle stages. As risks and changes occur, the plan and planning should be amended to recognize these and reflect the current position. As the plans will have been baselined (locked down) after initial sign-off, these changes would normally be managed by the project change process. Baselining a document effectively secures it from further change unless authorized via a change control process.
A useful revision aid to help remember the 16 sections of the IEEE 829 test plan is the acronym ‘SPACEDIRT’, each letter mapping to one or several sections of the test plan:
S scope (including test items, features to be tested and features not to be tested)
P people (including responsibilities, staff and training and approvals)
A approach
C criteria (including item pass/fail criteria and suspension and resumption requirements)
E environment needs
D deliverables (test)
I identifier and introduction (test plan)
R risks and contingencies
T testing tasks and schedule
Test-Planning Activities
During test planning various activities for an entire system or a part of a system have to be undertaken by those working on the plan. They include:
- Working with the project manager and subject matter experts' to determine the scope and the risks that need to be tested. As well identifying and agreeing the objectives of the testing, be they time, quality or cost focused, or in fact maybe a mixture of all three. The objectives will enable the test project to know when it has finished.
- Putting together the overall approach of testing (sometimes called the test strategy), ensuring that the test levels and entry and exit criteria are defined.
- Liaising with the project manager and making sure that the testing activities have been included within the software life-cycle activities such as:
2. development—the building of the code;
3. implementation—the activities surrounding implementation into a live environment.
- Working with the project to decide what needs to be tested, what roles are involved and who will perform the test activities, planning when and how the test activities should be done, deciding how the test results will be evaluated, and defining when to stop testing (exit criteria).
- Building a plan that identifies when and who will undertake the test analysis and design activities. In addition to the analysis and design activities test planning should also document the schedule for test implementation, execution and evaluation.
- Finding and assigning resources for the different activities that have been defined.
- Deciding what the documentation for the test project will be, e.g. which plans, how the test cases will be documented, etc.
- Defining the management information, including the metrics required and putting in place the processes to monitor and control test preparation and execution, defect resolution and risk issues.
- Ensuring that the test documentation generates repeatable test assets, e.g. test cases.
Entry criteria are used to determine when a given test activity can start. This could include the beginning of a level of testing, when test design and/or when test execution is ready to start.
Examples of some typical entry criteria to test execution (for example) may include:
- Test environment available and ready for use (it functions).
- Test tools installed in the environment are ready for use.
- Testable code is available.
- All test data is available and correct.
- All test design activity has completed.
Exit criteria are used to determine when a given test activity has been completed or when it should stop. Exit criteria can be defined for all of the test activities, such as planning, specification and execution as a whole, or to a specific test level for test specification as well as execution.
Exit criteria should be included in the relevant test plans.
Some typical exit criteria might be:
- All tests planned have been run.
- A certain level of requirements coverage has been achieved.
- No high-priority or severe defects are left outstanding.
- All high-risk areas have been fully tested, with only minor residual risks left outstanding.
- Cost—when the budget has been spent.
Exit criteria should have been agreed as early as possible in the life cycle; however, they can be and often are subject to controlled change as the detail of the project becomes better understood and therefore the ability to meet the criteria is better understood by those responsible for delivery.
Test Estimation
The syllabus details two test estimation approaches, metrics-based and expert based. The two approaches are quite different, the former being based upon data whilst the latter is a somewhat subjective approach.
The Metrics-Based Approach
This approach relies upon data collected from previous or similar projects. This kind of data might include:
- The number of test conditions.
- The number of test cases written.
- The number of test cases executed.
- The time taken to develop test cases.
- The time taken to run test cases.
- The number of defects found.
- The number of environment outages and how long on average each one lasted.
It is important that the actual costs and time for testing are accurately recorded. These can then be used to re validate and possibly update the metrics for use on the next similar project.
The Expert-Based Approach
This alternative approach to metrics is to use the experience of owners of the relevant tasks or experts to derive an estimate (this is also known as the Wide Band Delphi approach). In this context ‘experts’ could be:
- Business experts.
- Test process consultants.
- Developers.
- Technical architects.
- Analysts and designers.
- Anyone with knowledge of the application to be tested or the tasks involved in the process.
- Distribute a requirement specification to the task owners and get them to estimate their task in isolation. Amalgamate the individual estimates when received; build in any required contingency, to arrive at the estimate.
- Distribute to known experts who develop their individual view of the overall estimate and then meet together to agree on and/or debate the estimate that will go forward.
Many things affect the level of effort required to fulfill the test requirements of a project. These can be split into three main categories, as shown below.
- Product characteristics:
2. complexity of the final product;
3. the amount of non-functional requirements;
4. the security requirements (perhaps meeting BS 7799, the security standard);
5. how much documentation is required (e.g. some legislation-driven changes demand a certain level of documentation that may be more than an organization would normally produce);
the availability and quality of the test basis (e.g. requirements and specifications).
- Development process characteristics:
2. amount of budget available;
3. skills of those involved in the testing and development activity (the lower the skill level in development, the more defects could be introduced, and the lower the skill level in testing, the more detailed the test documentation needs to be);
4. which tools are being used across the life cycle (i.e. the amount of automated testing will affect the effort required).
- Expected outcome of testing such as:
2. test cases to be written.
Taking all of this into account, once the estimate is developed and agreed the test leader can set about identifying the required resources and building the detailed plan.
You may follow the complete series of Test Management articles here:
Test Management Risk And Testing
Software Test Organization
Software Test Approaches Test Strategies
Software Test Planning And Estimation
Software Test Progress Monitoring & Control
Software Testing Incident Management
Software Test Configuration Management
To see all articles of ISTQB-ISEB Foundation guide, see here:
Software Testing-ISTQB ISEB Foundation Guide
Post a Comment