Opioid Use Disorder (OUD) is a major public health crisis in the US affecting over 2.4 million Americans across all age groups and backgrounds. Previous studies have developed machine learning (ML) and deep learning (DL) models with strong predictive performance for OUD; however, bias related to sociodemographic features remains unaddressed. In this study, we propose a bias mitigation algorithm, based on equality of odds (EO), to mitigate ML model bias in OUD prediction. Our algorithm adjusts the classification threshold to minimize the difference between specificity and recall across sociodemographic groups. We develop a neural network (NN) model using stochastic gradient descent (NN-SGD) and Adam (NN-Adam) for OUD prediction and analyze bias by comparing the area under the curve (AUC) values. We implement our algorithm to mitigate bias related to gender, marital status, working condition, race, and income. The proposed algorithm, applied to NN-SGD, achieves significant improvements in bias reduction: 21.66% for gender, 1.48% for race, and 21.04% for income. Similarly, for the NN-Adam, the algorithm reduces bias by 16.96% (gender), 8.87% (marital status), 8.45% (working condition), 41.62% (race), and 0.20% (income). We also propose a fairness-aware weighted majority voting (WMV) classifier, achieving recall values of 85.37% and 92.68%, and accuracy values of 58.85% and 90.21% using the NN-SGD and NN-Adam, respectively. Furthermore, we evaluate the proposed methods using logistic regression and support vector machine classifiers with linear and radial basis function kernels, confirming the significant bias reduction and high performance of the WMV classifier for OUD prediction.