In this work, ensemble methods are presented and tested as universal ways to improve the performance of Mem-ristive Deep Neural Networks (MDNNs) with non-idealities. The Generalized Ensemble Method and Weighted Voting ensemble methods improve the accuracy of classification on the MNIST dataset by 6.5% and 6.6% respectively, thus showing that they are more effective than basic Ensemble Averaging which has been investigated before, as well as other methods such as Voting. Different weighting schemes for Weighted Voting were tested, and we present Algorithm 1 and 2, which are the theoretically and experimentally optimal weighting schemes respectively. Our work serves as a guideline for choosing ensemble methods for MDNNs.