UAT stands for User Acceptance Testing. It is a process to verify the application built/delivered works for business and system is accepted by client and users. During UAT, actual business/end users test the software to make sure it can handle required tasks in real-world scenarios, according to specifications. This testing can be executed by client in separate environment which can be like real time environment (replica of production) or can be actual real-time environment.
UAT is performed after System Testing is done by testing team and all the Critical and major defects should be fixed. It is important to confirm or take signoff from testing team before giving the system for UAT.
The main purpose of this testing is to validate the end to end business flow.
Why UAT is important?
- If the given system doesn’t work for external people such as customers, suppliers it can affect the goodwill of any company.
- It is possible that system can breaks the law or security norms and can create legal kiosk for product owner.
- The system may not meet key objectives which can affect the time to rebuild the system.
- UAT can give a clear picture of what should be expected as the outcome.
- Quality software builds confidence in the company that uses it, while it makes running the organization easier.
Who does UAT?
This testing is always performed by client or end users.
Pre-requisites of UAT:
- Business requirements must be available and should match the acceptance criteria.
- Application should be fully developed. In the case of Agile, when we follow Iteration, that Iteration should be completely done before system goes for UAT.
- Unit Testing, Integration Testing & System Testing should be completed.
- No Showstoppers, High, Medium defects should be open from System Testing Phase.
- Regression Testing should be completed with no major defects. All the reported defects should be fixed and tested before UAT.
- Sign off mail or communication from Testing Team should be given stating that the system is ready for UAT execution.
- Before UAT starts error like cosmetic error are acceptable (if agreed by client) but should be reported as known issues.
- The separate UAT environment like production should be ready to start UAT.
- User training along with user manual should be given to client to use the system
When UAT is performed?
It should be performed before the product goes live. In case of Agile multiple UAT phases can be performed based on Iteration or Drops.
User Acceptance Testing Process
One of the most important activities in the UAT is to identify and develop test scenarios. These test scenarios are derived from the following documents:
- Business use cases.
- Business flow.
- SRS Document.
- User stories document.
Deliverables for UAT:
1.UAT Test Plan:
It documents entry and exit criteria for UAT, Test scenarios and test cases approach and timelines of testing. It also includes dates, environment, actors(who), communication protocols, roles and responsibilities, templates, results and their analysis process, entry-exit criteria – all of this and anything else relevant will be found in the UAT test plan. Whether the QA team is participating, partially participating or not participating at all in the UAT, it is PMs job to plan this phase along with BA and tester and make sure everything is taken into consideration.
2.Test scenario or Test cases:
Identify the test scenarios with respect to high level business process and create test cases with clear test steps. Test Cases should sufficiently cover most of the UAT scenarios. Business Use cases are input for creating the test cases. QA team gives users a list of UAT test cases. UAT test cases are not different from our regular system test cases.
It is very important part where we should use live data so that we can also check the various aspects of data management and how system responds to larger number of data.
The test cases/test scenario given by team will be executed by end users or client. Which can give the information about the functionality covered with test scenario. Sometimes it also executes some relevant random tests. All bugs are logged in UAT Defect report with relevant comments.
We use UAT defect report for logging the bugs, it can also be done on tool (if client has access to tool). To see the format please check out this blog post.
We follow this document as UAT defect report. Here is the screen shot:
This template is divided into 2 sections:
One section is to be filled by Client which includes following columns:
Left to Right-
1.Defect #: Defect number or ID.
2.Date created: On which date the bug was raised or found.
3.Found by: Name of the person who raised the issue. It can be Client, QA or BA in or end user during UAT.
4.Module/ feature: In which module or feature they found the issue.
5.Defect type: Choose the defect type from following values-
- UI or UX
6.Defect Description: We need to mention here bug description: for eg: Application is not performing any action when user clicks on save button.
7.Priority: This column states how important is to fix this defect and contains following values:
- Critical: Critical impact to business functionality.
- High: Impact to business functionality but it’s not an emergency.
- Medium: Impact to business functionality but there is a work around.
- Low: Cosmetic with no impact to business functionality.
There is another section of the report which needs to be filled by QA team along with whole team.
8.Defect Category: Choose the any one value of Defect Category from the following values:
- Bug: Is this bug valid?
- Enhancement/ CR: Is it additional request raised by client for any feature.
- Clarification: Is it any query or doubt by the client?
- Known issue: Does this issue known to the client?
- Platform Limitation: Does this bug came because of limitations on the Platform.
- Invalid/Rejected: Is it bug invalid or rejected by team.
9.Validated by: Who validated or checked the defect in the system to make sure whether it is actual bug or no.
11.Defect available in QA: This column will specify answer with “Yes” or “No” values which states whether this Bug is available on QA site or No.
12.Estimated time to fix: Dev team needs to estimate the time to fix the issue. So, QA can estimate the time to retest.
13.Status: This column describes the status of Defect
- Open: Any bug which is new, or team has not considered it Yet.
- In Progress: Team is working on that defect.
- Fixed/Resolved: Team has fixed or resolved the issue
- Closed: Testing team has tested this issue and its closed.
- Cannot be fixed: This issue cannot be taken due to various reasons like limitations, lack of time etc.
14.Resolution: Dev team needs to specify what approach they took to fix this issue.
15.Resolution Date: On which date they fixed the issue.
16.Actual time to fix: How much time dev team took to fix the defect?
17.Root cause: Please mention the root cause of the defect from following values:
- Requirements: A defect which occurred because of any mismatch in requirement or lack of understanding of Requirements.
- Design: This issue came coz of poor system design.
- Build: This issue is related to software implementation or wrong coding.
- QA Testing: This issue is missed by Testing team.
- Deployment: This issue is related to incomplete or wrong Deployment.
View of complete UAT defect report:
We can configure this report format in the tools also, however many time clients will not have access to the tools. So, to make it faster we share in drives and keep updating it during the UAT.
As bug has life cycle, we do have life cycle when defect is reported by client:
Please zoom your browser screen to have better view of the diagram.
It’s important to follow the UAT process to get make sure system is good to go for live users. As a smooth software interaction in the business attracts more customers and the company’s reputation rises. The customers get better service and the company gets more customers.
About the author surbhi