Background: Second-line antiretroviral therapy (ART) was introduced in Henan Province in 2009. Most studies of this treatment strategy focus on drug resistance and treatment failure, not on mortality. To investigate the survival and effectors of mortality among patients with human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) who switched to second-line antiretroviral therapy (ART) in rural China. We conducted a retrospective cohort study of people living with HIV (PLHIV) who switched to second-line ART between May 1, 2010 to May 1, 2016. The data were analyzed using the Kaplan–Meier method and Cox proportional hazards models. Among 3331 PLHIV who were followed for 26988 person-years, 508 (15.3%) died and the mortality rate was 1.88/100 person-years. After adjusting for confounding factors multivariable Cox proportional hazards regression identified female (HR, 0.66; 95%CI, 0.55–0.79), age >50 years (HR, 2.69; 95%CI, 2.03–3.56), sig/window (HR, 1.26; 95%CI, 1.04–1.52) , educational status >6 years (HR, 0.78; 95%CI, 0.65–0.94), Chinese medicine(CM) (HR, 0.75; 95%CI, 0.52–0.96), liver injury (HR, 1.58; 95%CI, 1.19–2.10), CD4+ T cell count <200 cells/µl (HR, 1.94; 95%CI, 1.47-2.55), and CD4+ T cell count 200-350 cells/µl (HR, 1.37; 95%CI, 1.03–1.82) as independently variables associated with mortality. Conclusions: Our retrospective cohort study indicates that mortality among PLHIV who switched to second-line ART was lower compared with most other studies. However, the limitations of a retrospective cohort may have biased the data, so prospective studies should be carried out to confirm our primary results. The results of our study suggest that Chinese medicine therapy shows potential as a treatment for PLHIV.