Biological nitrogen fixation can be an important source of nitrogen in tropical forests that serve as a major CO2 sink. Extensive deforestation of the Amazon is known to influence microbial communities and the biogeochemical cycles they mediate. However, it is unknown how diazotrophs (nitrogen-fixing microorganisms) respond to deforestation and subsequent ecosystem conversion to agriculture, as well as whether they can recover in secondary forests that are established after agriculture is abandoned. To address these knowledge gaps, we combined a spatially explicit sampling approach with high-throughput sequencing of nifH genes. The main objectives were to assess the functional distance decay relationship of the diazotrophic bacterial community in a tropical forest ecosystem and to quantify the roles of various factors that drive the observed changes in the diazotrophic community structure. We observed an increase in local diazotrophic diversity (α-diversity) with a decrease in community turnover (β-diversity), associated with a shift in diazotrophic community structure as a result of the forest-to-pasture conversion. Both diazotrophic community turnover and structure showed signs of recovery in secondary forests. Changes in the diazotrophic community were primarily driven by the change in land use rather than differences in geochemical characteristics or geographic distances. The diazotroph communities in secondary forests resembled those in primary forests, suggesting that at least partial recovery of diazotrophs is possible following agricultural abandonment. IMPORTANCE The Amazon region is a major tropical forest region that is being deforested at an alarming rate to create space for cattle ranching and agriculture. Diazotrophs (nitrogen-fixing microorganisms) play an important role in supplying soil N for plant growth in tropical forests. It is unknown how diazotrophs respond to deforestation and whether they can recover in secondary forests that establish after agriculture is abandoned. Using high-throughput sequencing of nifH genes, we characterized the response of diazotrophs’ β-diversity and identified major drivers of changes in diazotrophs from forest-to-pasture and pasture-to-secondary-forest conversions. Studying the impact of land use change on diazotrophs is important for a better understanding of the impact of deforestation on tropical forest ecosystem functioning, and our results on the potential recovery of diazotrophs in secondary forests imply the possible restoration of ecosystem functions in secondary forests.