This section describes the integrated technique called ScriptAssure®, which is the recognition algorithm.
Property Value Weights
The success of an automated test is highly dependent on the robustness of the test script. Small changes often force development to adapt test scripts before running successfully. This reduces productivity because maintenance must be applied to the test scripts for a successful run.
For example, the old record and playback tools recorded based on recording graphical interaction. The actual x, y coordinates of a selection were stored in the script. When a button was moved or a different screen layout was applied to the application, the script became useless. It resulted in a lot of rework and crash maintenance to get scripts running.
Rational Functional Tester recognizes objects in the application under test. This means that objects can be moved around or changed graphically. Recognition is done based on all the properties of the object. Not only the visible label (the OK), but object properties are recorded. When one or more properties changes, Rational Functional Tester keeps running!
Sample 1: Login
An example is the button GO in a login screen, as shown in below Figure.
The base GO button has the following properties:
- ID = “GO”
- TYPE = “submit”
- NAME = “GO”
- VALUE = “GO”
When recording a script, the script contains the following line:
button_gOsubmit().click();
The button_gOsubmit is the object in the Script Explorer view, as shown in below Figure. A click is the action to be performed on that object.
Double-clicking the object in the tree opens the object map; here, you can interrogate the properties of the object, as shown in below Figure.
When the script is run to validate the script, it finds the object with all properties matching. The recognition algorithm is as easy as it is powerful; every mismatch results in 100 times the weight. The object with the minimum penalty points is selected.
One Property Change
Suppose the visible property GO is changed to Login, as shown in below Figure. An automatic test tool that recognizes only object visible labels fails, resulting in the test process stopping and an urgent need for maintenance.
Rational Functional Tester uses the same object or the one with minimal difference. In this case, Rational Functional Tester selects the Login, despite the difference in label, position, and format. The Reset button is not selected because there is a difference in all properties.
Two Properties Changed
Now with version 2, there are two properties changed, as shown in below Figure. Does Rational Functional Tester find the button?
With the default settings, Rational Functional Tester stores a message in the test log, as shown in below Figure. The Object Recognition is weak message indicates that Rational Functional Tester has found one object that is comparable. Rational Functional Tester uses this object to continue execution. Additionally, there is a recognition score (failing score) visible that is 19.000. The calculation can be derived by the rule that every mismatch results in 100 times the weight. This example shows a miss at ID (weight 90) and Value (weight 100), resulting in a 19.000. When no differences are found, this recognition score is 0.
Three Properties Changed
In the next version of the application, there are three properties changed, as shown in below Figure.
Now the object to be found and the object available in the application differ too much. A window opens to ask the user for advice, as shown in below Figure.
With the object browser, one can update the properties of the object to be found. The Playback Options become available just after starting a script. The previous interactive menu can be suppressed by deselecting the option Perform playback in interactive mode in the second screen of the Playback Options, as shown in below Figure.
If you update the properties of the object, you get a similar error message as shown in below Figure.
Rational Functional Tester does find an object candidate, but it has a failing score of 28500, as described in the log. This is above the value Last chance recognition score, which is set by default to 20000. If you increase this value to 50000, the script does find the Login button and provides a warning, as shown in below Figure.
Sample 2: Two Buttons
The previous examples clarify the behavior of ScriptAssure. The following example is somewhat more complex while we have two buttons, which are similar as shown in below Figure.
When the script is run, which of the buttons is selected, the “GO” (left) or the “GO” (right) button? Both have one change in a property. A part of the source of this menu is:
The object to be searched for is defined in the object map, as shown in below Figure.
Calculate the penalty points of the “Login” and the “GO” button, as shown in Table 1.1 and Table 1.2.
Table 1.1: Table for the First Go Button
Label | Property | Weight | Found | Result |
---|---|---|---|---|
.id | GO-TO-TAKE | 90 | No | 9000 |
.name | GO | 95 | Yes | 0 |
.type | Submit | 95 | Yes | 0 |
.value | GO | 100 | Yes | 0 |
Total | 9000 |
Table 1.2: Table for a Second Go Button
Label | Property | Weight | Found | Result |
---|---|---|---|---|
.id | GO | 90 | Yes | 0 |
.name | GO-NAME | 95 | No | 9500 |
.type | Submit | 95 | Yes | 0 |
.value | GO | 100 | Yes | 0 |
Total | 9500 |
If you replay with default ScriptAssure settings, you get the message AmbiguousRecognitionException, as shown in below Figure. This is because the two GOs are much the same.
When we decrease the Ambiguous recognition scores difference threshold to 200, for example, the script continues. So, the action attached to the GO-TO-TAKE button is used. If you are interested in verifying the properties of the object, the following is created in the object map, as shown in below Figure.
When you execute test script next time, the order number will not be 25. This results in penalty points, as shown in below Figure.
By setting the weight of accessibleContext.accessibleName and text to 0, there is a full match, but the recognition power is weaker. A better approach is to apply a regular expression as a value. This can be created via the contextual menu on the value. In this case, use a decimal definition \d+ as shown in Figure 1.55. The point is preceded by a backslash because the point is also a special character. For additional information about regular expressions, refer to Appendix B, “Regular Expressions in Rational Functional Tester.”
This sample is about the object to be searched for. For the changing value in the verification point, you can use a regular expression in the verification point.
ScriptAssure Playback Settings
The recognition and the warning levels can be influenced with settings at Window > Preferences > Functional Test > Playback > ScriptAssure.
The standard visualization gives two sliders to move in either direction, as shown in below Figure. In line with the error messages and the calculation described previously, you can use the advanced visualization.
If you click the Advance button, you get what’s shown in below Figure.
The ScriptAssure Advanced page has the following controls:
- Maximum acceptable recognition score— Indicates the maximum score an object can have to be recognized as a candidate. Objects with higher recognition scores are not considered as matches until the time specified in Maximum time to attempt to find Test Object has elapsed.
- Last chance recognition score— Indicates the maximum acceptable score an object must have to be recognized as a candidate, if Functional Tester does not find a suitable match after the time specified in Maximum time to attempt to find Test Object has elapsed. Objects with higher recognition scores are not considered.
- Ambiguous recognition scores difference threshold— Writes an AmbiguousRecognitionException to the log if the scores of top candidates differ by less than the value specified in this field. If Rational Functional Tester sees two objects as the same, the difference between their scores must be at least this value to prefer one object. You can override the exception by using an event handler in the script.
- Warn if accepted score is greater than— Writes a warning to the log if Rational Functional Tester accepts a candidate whose score is greater than or equal to the value in this field.
What are interesting settings? Defaults to start with because they work well. If you are in more of a dynamic user interface, you can increase the various values to acceptable levels. If you are doing acceptance testing, you can tighten the values and set the warn if option to 1. You always get a warning when something changes, but Rational Functional Tester continues to run.