View source for Acceptance testing

{{short description|Test to determine if the requirements of a specification or contract are met}}
{{Use mdy dates|date=October 2020}}
[[File:US Navy 090407-N-4669J-042 Sailors assigned to the air department of the aircraft carrier USS George H.W. Bush (CVN 77) test the ship's catapult systems during acceptance trials.jpg|thumb|right|Acceptance testing of an [[aircraft catapult]]]]
[[File:James Webb Primary Mirror.jpg|thumb|right|Six of the primary mirrors of the [[James Webb Space Telescope]] being prepared for acceptance testing]]
In [[engineering]] and its various [[fields of engineering|subdisciplines]], '''acceptance testing''' is a test conducted to determine if the requirements of a [[specification]] or [[contract]] are met. It may involve [[chemical test]]s, [[physical test]]s, or [[Performance test (assessment)|performance test]]s.<ref>{{Cite web |title=BPTS - Is Business process testing the best name / description. |url=https://sfia-online.org/en/sfia-8/change-requests/bpts-is-business-process-testing-the-best-name-description |access-date=2023-02-18 |website=SFIA |language=en}}</ref>

In [[systems engineering]], it may involve [[black-box testing]] performed on a [[system]] (for example: a piece of [[software system|software]], lots of manufactured mechanical parts, or batches of chemical products) prior to its delivery.<ref>{{Cite book |last=Black |first=Rex |url=https://archive.org/details/managingtestingp00rexb |title=Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing |date=August 2009 |publisher=Hoboken, NJ: Wiley |isbn=978-0-470-40415-7 |url-access=registration}}</ref>

In [[software testing]], the [[International Software Testing Qualifications Board|ISTQB]] defines ''acceptance testing'' as: {{Blockquote|text=Formal testing with respect to user needs, requirements, and business processes conducted to determine whether a system satisfies the [[acceptance criteria]]<ref>{{Cite web |date=2019-06-10 |title=acceptance criteria |url=https://innolution.com/resources/glossary/acceptance-criteria |publisher=Innolution, LLC}}</ref> and to enable the user, customers or other authorized entity to determine whether to accept the system.|source=Standard Glossary of Terms used in Software Testing<ref>{{Cite web |title=Standard Glossary of Terms used in Software Testing, Version 3.2: All Terms |url=https://glossary.istqb.org|access-date=November 23, 2020 |publisher=[[International Software Testing Qualifications Board|ISTQB]] |format=PDF}}</ref>{{rp|2}} |title=|author=}} The final test in the QA lifecycle, user acceptance testing, is conducted just before the final release to assess whether the product or application can handle real-world scenarios. By replicating user behavior, it checks if the system satisfies business requirements and rejects changes if certain criteria are not met.<ref>{{Cite web |date=2022-11-24 |title=User Acceptance Testing (UAT) - Software Testing |url=https://www.geeksforgeeks.org/user-acceptance-testing-uat/ |access-date=2024-05-23 |website=GeeksforGeeks |language=en-US}}</ref>

Some forms of acceptance testing are, [[user acceptance testing]] (UAT), end-user testing, [[operational acceptance testing]] (OAT), [[acceptance test-driven development]] (ATDD) and field (acceptance) testing. Acceptance criteria are the criteria that a system or component must satisfy in order to be accepted by a user, customer, or other authorized entity.<ref>{{Cite book |title=ISO/IEC/IEEE International Standard - Systems and software engineering |publisher=ISO/IEC/IEEE |year=2010 |pages=vol., no., pp.1–418}}</ref>

== Overview ==
Testing is a set of activities conducted to facilitate the discovery and/or evaluation of properties of one or more items under test.<ref name="ISO 29119 Part 1">{{Cite book |url=http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=45142 |title=ISO/IEC/IEEE 29119-1:2013 Software and Systems Engineering - Software Testing - Part 1: Concepts and Definitions |date=2013 |publisher=[[International Organization for Standardization|ISO]] |access-date=October 14, 2014}}</ref> Each test, known as a test case, exercises a set of predefined test activities, developed to drive the execution of the test item to meet test objectives; including correct implementation, error identification, quality verification, and other valued details.<ref name="ISO 29119 Part 1" /> The test [[deployment environment|environment]] is usually designed to be identical, or as close as possible, to the anticipated production environment. It includes all facilities, hardware, software, firmware, procedures, and/or documentation intended for or used to perform the testing of software.<ref name="ISO 29119 Part 1" />

UAT and OAT test cases are ideally derived in collaboration with business customers, business analysts, testers, and developers. These tests must include both business logic tests as well as operational environment conditions. The business customers (product owners) are the primary [[project stakeholder|stakeholders]] of these tests. As the test conditions successfully achieve their acceptance criteria, the stakeholders are reassured the development is progressing in the right direction.<ref name="ISO 29119 Part 4">{{Cite book |url=http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=60245 |title=ISO/IEC/IEEE 29119-4:2013 Software and Systems Engineering - Software Testing - Part 4: Test Techniques |date=2013 |publisher=[[International Organization for Standardization|ISO]] |access-date=October 14, 2014}}</ref>

* User acceptance test (UAT) criteria (in [[agile software development]]) are usually created by business customers and expressed in a [[Domain specific language|business domain language]]. These are high-level tests to verify the completeness of a [[user story]] or stories 'played' during any sprint/iteration. 
* Operational acceptance test (OAT) criteria (regardless of using agile, iterative, or sequential development) are defined in terms of [[Functional requirement|functional]] and [[non-functional requirement]]s; covering key quality attributes of [[software testing|functional stability]], [[portability testing|portability]], and [[reliability theory|reliability]].

== Process ==
The acceptance test suite may need to be performed multiple times, as all of the test cases may not be executed within a single test iteration.<ref name="ISO 29119 Part 2">{{Cite book |url=http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=45142 |title=ISO/IEC/IEEE 29119-2:2013 Software and Systems Engineering - Software Testing - Part 2: Test Processes |date=2013 |publisher=[[International Organization for Standardization|ISO]] |access-date=May 21, 2014}}</ref>

The acceptance test suite is run using predefined acceptance test procedures to direct the testers on which data to use, the step-by-step processes to follow, and the expected result following execution. The actual results are retained for comparison with the expected results.<ref name="ISO 29119 Part 2" /> If the actual results match the expected results for each test case, the test case is said to pass. If the quantity of non-passing test cases does not breach the project's predetermined threshold, the test suite is said to pass. If it does, the system may either be rejected or accepted on conditions previously agreed between the sponsor and the manufacturer.

The anticipated result of a successful test execution:
* test cases are executed, using predetermined data
* actual results are recorded
* actual and expected results are compared, and
* test results are determined.

The objective is to provide confidence that the developed product meets both the functional and non-functional requirements. The purpose of conducting acceptance testing is that once completed, and provided the acceptance criteria are met, it is expected the sponsors will sign off on the product development/enhancement as satisfying the defined requirements (previously agreed between business and product provider/developer).

== User acceptance testing ==

User acceptance testing (UAT) consists of a process of verifying that a solution works for the user.<ref>{{Cite book |last=Cimperman |first=Rob |title=UAT Defined: A Guide to Practical User Acceptance Testing |publisher=Pearson Education |year=2006 |isbn=9780132702621 |pages=Chapter 2}}</ref> It is not [[system testing]] (ensuring software does not crash and meets documented requirements) but rather ensures that the solution will work for the user (i.e. tests that the user accepts the solution); software vendors often refer to this as "Beta testing".

This testing should be undertaken by the intended [[end user]], or a [[subject-matter expert]] (SME), preferably the owner or client of the solution under test, and provide a summary of the findings for confirmation to proceed after trial or review. In [[Software development process|software development]], UAT as one of the final stages of a project often occurs before a client or customer accepts the new system. Users of the system perform tests in line with what would occur in real-life scenarios.<ref>{{Cite book |last1=Goethem |first1=Brian |last2=van Hambling |first2=Pauline |title=User acceptance testing : a step-by-step guide |publisher=BCS Learning & Development Limited |year=2013 |isbn=9781780171678}}</ref>

The materials given to the tester must be similar to the materials that the end user will have. Testers should be given real-life scenarios such as the three most common or difficult tasks that the users they represent will undertake.<ref>{{Cite web |date=2021-08-02 |title=2.6: Systems Testing |url=https://eng.libretexts.org/Courses/Prince_Georges_Community_College/INT_2840%3A_Systems_Analysis_and_Project_Management/02%3A_Information_Systems_Development_And_RFP/2.06%3A_Systems_Testing |access-date=2023-02-18 |website=Engineering LibreTexts |language=en}}</ref>

The UAT acts as a final verification of the required business functionality and proper functioning of the system, emulating real-world conditions on behalf of the paying client or a specific large customer. If the software works as required and without issues during normal use, one can reasonably extrapolate the same level of stability in production.<ref>{{Cite book |last=Pusuluri |first=Nageshwar Rao |title=Software Testing Concepts And Tools |publisher=Dreamtech Press |year=2006 |isbn=9788177227123 |page=62}}</ref>

User tests, usually performed by clients or by end-users, do not normally focus on identifying simple cosmetic problems such as spelling errors, nor on [[Software bug|showstopper]] defects, such as [[crash (computing)|software crashes]]; testers and developers identify and fix these issues during earlier [[unit testing]], [[integration testing]], and system testing phases.

UAT should be executed against test scenarios.<ref>{{Cite web |date=2022-04-25 |title=Get Reliable Usability and Avoid Risk with These Testing Scenarios |url=https://www.panaya.com/blog/testing/testing-scenarios/ |access-date=2022-05-11 |website=Panaya |language=en-US}}</ref><ref>{{Cite web |last=Elazar |first=Eyal |date=2018-04-23 |title=What is User Acceptance Testing (UAT) - The Full Process Explained |url=https://www.panaya.com/blog/testing/what-is-uat-testing/ |access-date=2023-02-18 |website=Panaya |language=en-US}}</ref> Test scenarios usually differ from System or Functional test cases in that they represent a "player" or "user" journey. The broad nature of the test scenario ensures that the focus is on the journey and not on technical or system-specific details, staying away from "click-by-click" test steps to allow for a variance in users' behaviour. Test scenarios can be broken down into logical "days", which are usually where the actor (player/customer/operator) or system (backoffice, front end) changes.<ref>{{Cite journal |last=Wysocka |first=Emilia M. |last2=Page |first2=Matthew |last3=Snowden |first3=James |last4=Simpson |first4=T. Ian |date=2022-12-15 |title=Comparison of rule- and ordinary differential equation-based dynamic model of DARPP-32 signalling network |url=https://peerj.com/articles/14516 |journal=PeerJ |language=en |volume=10 |at="Table 1: The specifications of the ODE and RB models can be broken down into elements, the number of which can be compared." |doi=10.7717/peerj.14516 |issn=2167-8359 |pmc=9760030 |pmid=36540795 |doi-access=free }}</ref>

In industry, a common UAT is a factory acceptance test (FAT). This test takes place before the installation of the equipment. Most of the time testers not only check that the equipment meets the specification but also that it is fully functional. A FAT usually includes a check of completeness, a verification against contractual requirements, a proof of functionality (either by simulation or a conventional function test), and a final inspection.<ref>{{Cite web |title=Factory Acceptance Test (FAT) |url=http://www.tuv.com/en/corporate/business_customers/materials_testing_and_inspection/supply_chain_services/factory_acceptance_test/factory_acceptance_test.jsp |url-status=dead |archive-url=https://archive.today/20130204215825/http://www.tuv.com/en/corporate/business_customers/materials_testing_and_inspection/supply_chain_services/factory_acceptance_test/factory_acceptance_test.jsp |archive-date=February 4, 2013 |access-date=September 18, 2012 |publisher=TÜV Rheinland }}</ref>
The results of these tests give clients confidence in how the system will perform in production. There may also be legal or contractual requirements for acceptance of the system.

==Operational acceptance testing==
{{Main|Operational acceptance testing}}

[[Operational acceptance testing]] (OAT) is used to conduct operational readiness (pre-release) of a product, service or system as part of a [[quality management system]]. OAT is a common type of non-functional [[software testing]], used mainly in [[software development]] and [[software maintenance]] projects. This type of testing focuses on the operational readiness of the system to be supported, and/or to become part of the production environment.<ref>{{Cite news |last=Vijay |date=2018-02-02 |title=What is Acceptance Testing (A Complete Guide) |language=en-US |work=Software Testing Help |url=https://www.softwaretestinghelp.com/what-is-acceptance-testing/ |access-date=2023-02-18}}</ref>

== Acceptance testing in extreme programming ==
{{Main|Extreme programming}}

Acceptance testing is a term used in [[agile software development]] methodologies, particularly [[extreme programming]], referring to the [[functional testing]] of a [[user story]] by the software development team during the implementation phase.<ref>{{Cite web |title=Introduction to Acceptance/Customer Tests as Requirements Artifacts |url=http://www.agilemodeling.com/artifacts/acceptanceTests.htm |access-date=December 9, 2013 |website=agilemodeling.com |publisher=Agile Modeling}}</ref>

The customer specifies scenarios to test when a user story has been correctly implemented. A story can have one or many acceptance tests, whatever it takes to ensure the functionality works. Acceptance tests are black-box system tests. Each acceptance test represents some expected result from the system. Customers are responsible for verifying the correctness of the acceptance tests and reviewing test scores to decide which failed tests are of highest priority. Acceptance tests are also used as [[Regression testing|regression tests]] prior to a production release. A user story is not considered complete until it has passed its acceptance tests. This means that new acceptance tests must be created for each iteration or the development team will report zero progress.<ref>{{Cite web |last=Wells |first=Don |title=Acceptance Tests |url=http://www.extremeprogramming.org/rules/functionaltests.html |access-date=September 20, 2011 |publisher=Extremeprogramming.org}}</ref>

{{Expand section|date=May 2008}}

== Types of acceptance testing ==
{{unreferenced section|date=March 2015}}
Typical types of acceptance testing include the following

; [[Verification and validation#Activities|User acceptance testing]]:This may include factory acceptance testing (FAT), i.e. the testing done by a vendor before the product or system is moved to its destination site, after which site acceptance testing (SAT) may be performed by the users at the site.<ref>{{Cite news |last=Prasad |first=Durga |date=March 29, 2012 |title=The Difference Between a FAT and a SAT |work=Kneat.com |url=http://www.kneat.com/2012/03/29/the-difference-between-a-fat-and-a-sat/ |access-date=July 27, 2016 |archive-date=June 16, 2017 |archive-url=https://web.archive.org/web/20170616183242/http://www.kneat.com/2012/03/29/the-difference-between-a-fat-and-a-sat/ |url-status=dead }}</ref>
; [[Operational acceptance testing]]:Also known as operational readiness testing, this refers to the checking done to a system to ensure that processes and procedures are in place to allow the system to be used and maintained. This may include checks done to back-up facilities, procedures for disaster recovery, training for end users, maintenance procedures, and security procedures.<ref>{{Cite web |last=Turner |first=Paul |date=2020-10-05 |title=Operational Readiness |url=https://commissioningandstartup.com/operational-readiness/ |access-date=2023-02-18 |website=Commissioning and Startup |language=en-US}}</ref>
; Contract and regulation acceptance testing
: In contract acceptance testing, a system is tested against acceptance criteria as documented in a contract, before the system is accepted. In regulation acceptance testing, a system is tested to ensure it meets governmental, legal and safety standards.<ref>{{Cite news |last=Brosnan |first=Adeline |date=2021-01-12 |title=Acceptance Testing in Information Technology Contracts {{!}} LegalVision |language=en-US |work=LegalVision |url=https://legalvision.com.au/acceptance-testing-information-technology-contracts/ |access-date=2023-02-18}}</ref>

; Factory acceptance testing
: Acceptance testing conducted at the site at which the product is developed and performed by employees of the supplier organization, to determine whether a component or system satisfies the requirements, normally including hardware as well as software.<ref>{{Cite web |title=ISTQB Standard glossary of terms used in Software Testing |url=http://glossar.german-testing-board.info/ |access-date=March 15, 2019 |archive-date=November 5, 2018 |archive-url=https://web.archive.org/web/20181105182224/http://glossar.german-testing-board.info/ |url-status=dead }}</ref>

; Alpha and beta testing
: Alpha testing takes place at developers' sites, and involves testing of the operational system by internal staff, before it is released to external customers. Beta testing takes place at customers' sites, and involves testing by a group of customers who use the system at their own locations and provide feedback, before the system is released to other customers. The latter is often called "field testing".<ref>{{Cite web |last=Hamilton |first=Thomas |date=2020-04-03 |title=Alpha Testing Vs Beta Testing – Difference Between Them |url=https://www.guru99.com/alpha-beta-testing-demystified.html |access-date=2023-02-18 |website=www.guru99.com |language=en-US}}</ref>

==Acceptance criteria==
According to the [[Project Management Institute]], '''acceptance criteria''' is a "set of conditions that is required to be met before deliverables are accepted."{{sfn|Project Management Institute|2021|loc=§Glossary Section 3. Definitions}}
Requirements found in acceptance criteria for a given component of the system are usually very detailed.{{sfn|Project Management Institute|2021|loc=§2.6.2.1 Requirements}}

== List of acceptance-testing frameworks ==
* [[Concordion]], [[Specification by example]] (SbE) framework
** Concordion.NET, acceptance testing in .NET
* [[Cucumber (software)|Cucumber]], a [[behavior-driven development]] (BDD) acceptance test framework
** Capybara, Acceptance test framework for Ruby web applications
** Behat, BDD acceptance framework for PHP
** Lettuce, BDD acceptance framework for Python
* [[Cypress (software)|Cypress]]
* [[Fabasoft app.test]] for automated acceptance tests
* [[Framework for Integrated Test]] (Fit)
** [[FitNesse]], a [[Fork (software development)|fork]] of Fit
* [[Gauge (software)]], Test Automation Framework from [[Thoughtworks]]
* [[iMacros]]
* ItsNat Java Ajax web framework with built-in, server based, functional web testing capabilities.
* [[Maveryx]] Test Automation Framework for functional testing, regression testing, GUI testing, data-driven and codeless testing of Desktop and Web applications.
* [[Mocha (JavaScript framework)|Mocha]], a popular web acceptance test framework based on Javascript and Node.js
* [[Playwright (software)]]
* [[Ranorex]]
* [[Robot Framework]]
* [[Selenium (software)|Selenium]]
* [[Specification by example]] (Specs2)
* [[Watir]]

== See also ==

* [[Acceptance sampling]]
* [[Conference room pilot]]
* [[Development stage]]
* [[Dynamic testing]]
* [[Engineering validation test]]
* [[Grey box testing]]
* [[Test-driven development]]
* [[White box testing]]
* [[Functional testing (manufacturing)]]

== References ==
{{Reflist}}

==Sources==
*{{Cite book |title=A guide to the project management body of knowledge (PMBOK guide)|date=2021|publisher=Project Management Institute|isbn=978-1-62825-664-2
|edition=7th|location=Newtown Square, PA|ref={{SfnRef|Project Management Institute|2021}} }}

== Further reading ==
* {{Cite book |last1=Hambling |first1=Brian |title=User Acceptance Testing: A Step by Step Guide |last2=van Goethem |first2=Pauline |publisher=BCS Learning and Development Ltd |year=2013 |isbn=978-1-78017-167-8 |location=Swindon}}

== External links ==

* ''[http://testingguidance.codeplex.com Acceptance Test Engineering Guide] {{Webarchive|url=https://web.archive.org/web/20171223193709/http://testingguidance.codeplex.com/ |date=December 23, 2017 }}'' by [http://msdn.com/practices Microsoft patterns & practices]
* "[http://www.methodsandtools.com/archive/archive.php?id=23 Using Customer Tests to Drive Development]" from ''[http://www.methodsandtools.com/ Methods & Tools]''

{{Software testing}}
{{Authority control}}

{{DEFAULTSORT:Acceptance Testing}}
[[Category:Facilities engineering]]
[[Category:Software testing]]
[[Category:Hardware testing]]
[[Category:Procurement]]
[[Category:Agile software development]]