Use Case Testing Description

Use case testing is the act of acting out specific test cases with the intent of exercising the whole system (or specific workflows) on a transaction-by-transaction basis, from start to finish, in order to uncover areas of nonconformance with specified acceptance criteria. In the case of Use case testing by AFixt the goal is to identify accessibility challenges in the performance of those use cases while using assistive technologies.

A use case is a description of a particular use (aka workflow) of the system as defined by a specified actor. The use case describes, in step-by-step detail, the specific actions the actor must perform in order to achieve a desired outcome. In use case testing performed by AFixt, the results of the testing provide insight into exactly which steps could or could not be completed, challenges that arose during testing, and possible causes for those challenges. In doing so, we are able to provide detailed insight into how accessible a system is for real users.

This type of testing has a number of important differences between other types of testing. Like automated testing, it is not a replacement for other types of testing but rather a component of a robust testing methodology.

In light of the above, Use case testing can be regarded as a mid-point between manual testing and usability testing.

Successful Use case testing requires two things: good test cases and good test execution. The true value of use case testing is in the detailed feedback provided to explain what went wrong, where it went wrong, and (hopefully) why it went wrong. When writing the test cases, always consider shorter test cases rather than long ones.

A good test case

• Test a single, well-defined process.
• Identifies all necessary preconditions.
• Has a clearly defined starting point.
• Has a clearly defined outcome or goal
• Provides detailed instructions necessary to perform the case.
• Uses action words to define the steps

A bad test case is one in which any of the above items are not true. The most common causes of bad use case testing are:

• Attempting to test too many processes at once
• Testing something which has no concretely definable goal
• Missing or poorly documented steps

Defining success or failure of a use case depends, first and foremost, on having a clearly defined outcome, such as "Register for a new account". The judgment of "pass" or "fail", therefore, is dependent upon whether the tester can successfully register for a new account or not.

• Start Location: /register.php
• Preconditions: User must not already be logged in; email address must not exist in database
• Goal: Register for a new account

Steps

1. Access the start location

2. Locate the heading "Register"

3. Locate the field labeled "E-mail Address"

4. Gain focus on the field labeled "E-mail Address"

5. Enter email address ([email protected]) in the field labeled 'E-mail Address'

6. Locate the field labeled "Password"

7. Enter a password (password1) in the field labeled 'Password'

8. Locate the button labeled "Sign Up"

9. Activate the button labeled 'Sign up'. Form is submitted.

10. Verify message appears stating, "You have been successfully registered..."

11. Verify email is sent with a confirmation link.

The example above demonstrates each of the above concepts relating to the structure and wording of a good test case. From there, testing is relatively simple: Walk through each of the steps, noting which steps had problems, whether the test case was successful or not and if not, where the failure occurred.

Note: unlike usability testing, the tester should attempt to continue the testing. The goal of use case testing isn’t just to determine if the case is successful or not but also to provide valuable accessibility feedback. To the extent that they can do so, testers should attempt to continue testing, even if it requires assistance. In such cases the failure should be noted as such, but testing continued.

Along each step of the way, critical notes should be taken to identify what should be given further scrutiny by development staff to determine the cause of the issues.

The following is an example of logging test results for the aforementioned test case

• Start Location: /register.php
• Preconditions: User must not already be logged in; email address must not exist in database
• Goal: Register for a new account

Steps

Note in the above that each step relating to filling in form fields or activating controls actually has two steps: "Locate..." and "Enter..."/ "Activate..." This level of granularity is important because each of these is a potential failure point and may also have distinct error causes. For instance, a tester may be able to locate the "Sign Up" button but may not be able to activate it using a mouse. Or the converse may be true. Separating these actions allows for more clarity in reporting and provides important diagnostic information to developers.

In addition to the above notes, the test case should be graded with one of the following values:

• Pass: The test case could be completed with little-to-no difficulty

• Pass w/ conditions: The test case could be completed though doing so was difficult or frustrating.

• Fail: The test case could not be completed. This is also the grade to be used for those cases requiring assistance

In order to maintain quality, clarity, and reliability, the following guidelines should be followed when writing the test case steps:

Use the following keywords when writing steps:

• Locate: This keyword is to be used in cases where the tester should look for a specific piece of content or a specific control.
• Gain Focus On: This keyword is to be used in cases where the tester must interact with a control. This step is distinct from the others in that gaining focus on a control may be the critical failure in the test case while the others may be successful after you’ve gotten focus.
• Enter: This keyword is to be used when instructing the tester to fill in information into a text input or textarea
• Select: This keyword is to be used when instructing the tester to make a choice from a series of radio buttons, select elements, or checkboxes.
• Activate: This keyword is to be used when instructing the tester to activate a button or a link.
• Verify: This keyword is to be used when defining a step or outcome that is critical to determining success of a step or of the test case as a whole.

When writing test cases, ensure that you’re clearly identifying all controls so that they can be found easily. Typically you will refer to the controls by their visible on-screen label, i.e. "The Sign Up Button" rather than "the button" or "the 3rd link in the menu". Doing so will ensure that the user can follow the proper steps.

Testing accessibility with use cases would be incomplete if we didn’t also test what happens when something goes wrong. A system’s ability to not only cope with user error but also do so accessibly is critical to determining its overall accessibility. To do so, we need to perform tests of alternate scenarios. These may or may not be "extensions" to existing cases. To do so, we perform tests aimed at triggering errors or exceptions.

• Start Location: /register.php
• Preconditions: none
• Goal: trigger error handling while registering for an account

Steps

1. Access the start location

2. Enter email address in the field labeled 'E-mail Address' (ensure you enter a valid email address that does not already exist in the system)

3. Enter a password only 3 characters long in the field labeled 'Password'

4. Activate the button labeled 'Sign up'

5. Verify global error message appears: "Errors are preventing successful submission of this form."

6. Verify field-level error message appears above password field.

In the above example, the same use case is performed in a way that is specifically aimed at causing the "normal" case to fail. What is being tested is how accessibly failure is handled.