Online experimentation helps companies evaluate product ideas and can accelerate organizational learning. However, online experiments can be costly and time consuming. Consequently, teams must carefully evaluate what experiments to prioritize. To triage product ideas before online experimentation, Pinterest developed Offline Replay Experimentation, a framework that applies counterfactual serving simulation and reward estimation. Counterfactual serving simulation simulates the experience users would have had if a change was deployed; reward estimation calculates lift by estimating the reward of the counterfactual serving results for each experiment variant. In this post, Maxine Qian describes the methodology for reward estimation and walks through an example wherein the Offline Replay Experimentation framework is applied to predict the performance of Home feed ranking and blending models.