This Supporting Information includes the simulation code on the training of the nonlinear  memristor neural network using the self-adaptive learning method discussed in the main text. The demonstrated network is LeNet-5 and can be easily extended to be larger scale networks. The dataset in this work is the typical Modified National Institute of Standards and Technology handwriting digits dataset that is available at http://yann.lecun.com/exdb/mnist/.