Abstract—While deep learning-based methods have gained popularity and have made remarkable progress in remote sensing (RS) image change detection (CD), the limited amount of available data hinders the performance of most supervised methods. The CD networks transferred or derived from other fields can be fronted with a weak generalization capability. Developing a universal benchmark for performance evaluations based on the available datasets is urgent. To address these problems, we proposed a lightweight network, termed 3M-CDNet, which only requires about 3.12 M parameters. The lighter the network, the easier it is to train and alleviate overfitting the limited amount of data, resulting in a better generalization capability. 3M-CDNet has a flexible modular design that achieves performance improvements by incorporating plug-and-play modules. 3M-CDNet gains accuracy improvements in two ways: (1) the application of deformable convolutions (DConv) in the backbone network to gain a good geometric transformation modeling capacity for CD and (2) the application of an effective two-level feature fusion strategy to enhance the feature representation capacity. 3M-CDNet gains a good generalization capacity by incorporating effective “tricks” to alleviate overfitting, in which online data augmentation (Online DA) is applied to increase the diversity of the training samples, and Dropout regularization is applied in the classifier. Extensive ablation studies have proved the effectiveness of the core components. Experiment results suggest that 3M-CDNet outperforms state-of-the-art methods on several optical RS datasets and serves as a new universal benchmark. Specifically, 3M-CDNet achieves the best F1-score, i.e., LEVIR-CD (0.9161), Season-Varying (0.9473), and DSIFN (0.7031).