User Acceptance & Usability Testing

You have a service you want to test to make sure it functions as designed. You are hoping to get feedback from your potential users. You want to make sure your service meets your business and the users' needs.  

How can you answer all of these questions about your service? What kind of testing can you use? A user acceptance test? A usability test? Or both?

Let’s find out!  

What is User Acceptance Testing (UAT)?

User acceptance testing is a method used to test materials or a service to ensure it has the necessary requirements to function properly

The “User” in UAT is the Business

For example, FEMA would define the requirements and have their development team conduct user acceptance testing to make sure their disaster relief form has the necessary information and data collection points to help those who have been impacted by a disaster.

By doing user acceptance testing, FEMA is checking their disaster relief form to confirm that it meets the business requirements before release.

Breaking Down the Process:

1. The business defines and approves the requirements and criteria for the service to function properly.
2. They create a script or guide to test the service to check its functionality.
3. Once testing is completed and the service meets requirements, the service is released for public use.

User acceptance testing is a vital method that is utilized throughout the lifecycle of a service; however, it cannot be the only method you use.

What is Usability Testing (UT)?

A usability test provides the users insights that UAT does not and increases our maturity in delivering more user-focused solutions.  

For example, USCG would conduct usability testing when building a new online application form. They would recruit potential users and ask them to test a prototype of the online application form to uncover any usability issues and opportunities to make it more relevant for their intended user. Based on the feedback from users, the USCG could make changes to the online application form to resolve uncovered issues and better meet the user's needs.

A usability test gives you important information about how real users interact with your materials, helping you to make informed decisions about how to improve existing services and check the effectiveness, efficiency, and accessibility of new services before you launch them out into the world.

Breaking Down the Process:

1. Define Goals and Design the Test: Identify tasks you want to observe people doing and create target goals you want to meet while completing the usability test.
2. Recruit Users: Find 3-5 volunteers to participate in individual usability testing sessions.
3. Create Moderator’s Guide: Create a checklist to follow to give the participant instructions to test a service.
4. Conduct Test: Ask the participant to complete real-life tasks within the service to test how easy or difficult it is for users to reach their own goals. Observe and note where users experience confusion, frustration, or ask questions. Capture how they interact with the service and record feedback. (In-person or remote)
5. Synthesize Findings: Organize research data to identify patterns in where the design supports users well and where it does not and analyze why.
6. Iterate Service: Based on participant feedback, update the service to better serve the users.

Usability testing should be repeated throughout the lifecycle of the service to continuously iterate the service to meet the user's needs. You can think of each test as a step toward providing a more effective and equitable service.

Why We Should Use Both

Using only one method can lead to not meeting the business’s needs and/or users having usability issues.

Take This Example for Instance:

A button on a website may allow form data to be submitted, but it may be in a place on the page in the user interface that makes it hard for the user to see or click the button.

In this case, it will likely pass a user acceptance test because the button is on the page, and it functions properly. By also completing usability testing this issue can be found. If the business completes a user acceptance test and a usability test, they catch this issue by using user feedback.

To avoid any confusion or frustration from our users, we should use both methods, user acceptance testing and usability testing to help us learn about the effectiveness of our services and provide value towards delivering intuitive, human-centered services.

In Summary

When applying usability testing to a service, we are actively seeking user feedback to make improvements to our services. With each improvement we make, we are creating a better customer experience across each of our DHS Components.

By making user acceptance testing and usability testing regular practices within your program or service team, you can help your organization enhance your customer’s experience. As you deliver effective, efficient, and equitable services you are ensuring your Component is reducing its administrative and public burden hours associated with its key services.