Summary.
In recent years the use of online A/B testing has skyrocketed, fueled by a growing appreciation of its value and by the relatively low cost of technology for conducting it. Today digital firms and, increasingly, conventional companies each run tens of thousands of online experiments annually measuring whether “A,” a control (usually the current approach), is inferior to “B,” a proposed improvement to a product, service, or offer. By quickly revealing users’ reactions to modifications, the experiments help firms identify the best ways to update digital products and create new ones. Because they push innovations out to a small, randomly selected group before they’re released to everyone, the tests also lessen the risk of unintended adverse side effects. And their unique ability to objectively measure the impact of a change enables firms to disentangle any growth in revenues, engagement, or other key business metric that improvements produce from growth that would have happened anyway. That vital information allows firms to spot opportunities and accurately assess their return on investment.