Background Air emissions from concentrated animal feeding operations (CAFO) have been associated with respiratory and allergic symptoms among farm workers, primarily on swine farms. Despite the increasing prevalence of CAFOs, few studies have assessed respiratory health implications among residents living near CAFOs and few have looked at the health impacts of dairy CAFOs. Objectives The goal of this study was to examine objective and subjective measures of respiratory and allergic health among rural residents living near dairy CAFOs in a general population living in the Upper Midwest of the United States. Methods Data were from the 2008–2016 Survey of the Health of Wisconsin (SHOW) cohort (n = 5338), a representative, population based sample of rural adults (age 18+). The association between distance to the nearest CAFO and the prevalence of self-reported physician-diagnosed allergies, asthma, episodes of asthma in the last 12 months, and asthma medication use was examined using logistic regression, adjusting for covariates and sampling design. Similarly, the association between distance to the nearest CAFO and lung function, measured using spirometry, was examined using multivariate linear regression. Restricted cubic splines accounted for nonlinear relationships between distance to the nearest CAFO and the aforementioned outcomes. Results Living 1.5 miles from a CAFO was associated with increased odds of self-reported nasal allergies (OR = 2.08; 95% CI: 1.38, 3.14), lung allergies (OR = 2.72; 95% CI: 1.59, 4.66), asthma (OR = 2.67; 95% CI: 1.39, 5.13), asthma medication (OR = 3.31; 95% CI: 1.65 6.62), and uncontrolled asthma, reported as an asthma episode in last 12 months (OR = 2.34; 95% CI: 1.11, 4.92) when compared to living 5 miles from a CAFO. Predicted FEV1 was 7.72% (95% CI: −14.63, −0.81) lower at a residential distance 1.5 miles from a CAFO when compared with a residence distance of 3 miles from a CAFO. Conclusions Results suggest CAFOs may be an important source of adverse air quality associated with reduced respiratory and allergic health among rural residents living in close proximity to a CAFO.