joint work with Nathan Kallus
Applications often involve solving thousands of potentially unrelated stochastic optimization problems, each with limited data. Intuition suggests decoupling and solving these problems separately. We propose a novel data-pooling algorithm that combines problems and outperforms decoupling, even if data are independent. Our method does not require strong distributional assumptions and applies to constrained, possibly non-convex problems. As the number of problems grows large, our method learns the optimal amount to pool, even if the expected amount of data per problem is bounded and small.
joint work with Jackie Baek, Vivek Farias, Deeksha Sinha
The cost of DNA sequencing has recently fallen to the point that an affordable blood test for early-stage cancer is nearly feasible. What remains is a massive variable selection problem. We propose an efficient algorithm, based on a decomposition at the gene level, that scales to full genomic sequences across thousands of patients. We contrast our selected variables against DNA panels from two recent, high-profile studies and demonstrate that our own panels achieve significantly higher sensitivities at the same cost, along with accurate discrimination between cancer types for the first time.
joint work with Yi-Chun Chen
Most choice models assume that customers obey the weak rationality property, but increasing empirical evidence suggests that customers. We propose a new model, the decision forest model, which models customers via a distribution over trees. We characterize its representational power, study the model complexity needed to fit a data set and present a practical estimation algorithm based on randomization and linear optimization. We show using real transaction data that this approach yields improvements in predictive accuracy over mainstream models when customers behave irrationally.