To enable uncertainty estimation and improve robustness, some researchers have proposed using a distribution over neural networks. Although these methods achieve strong uncertainty and robustness performance with negligible additional parameters, they usually have high runtime costs because they require 4-10 forward passes for prediction. However, Havasi et al. show that by concurrently training multiple independent subnetworks within one network (using a multi-input, multi-output configuration) they can improve uncertainty, estimation, and robustness in a single forward pass.