Age-related changes in cognition, brain morphology, and behavior are exhibited in several primate species. Baboons, like humans, naturally develop Alzheimer's disease-like pathology and cognitive declines with age and are an underutilized model for studies of aging. To determine age-related differences in gray matter covariation of 89 olive baboons (Papio anubis), we used source-based morphometry (SBM) to analyze data from magnetic resonance images. We hypothesized that we would find significant age effects in one or more SBM components, particularly those which include regions influenced by age in humans and other nonhuman primates (NHPs). A multivariate analysis of variance revealed that individual weighted gray matter covariation scores differed across the age classes. Elderly baboons contributed significantly less to gray matter covariation components including the brainstem, superior parietal cortex, thalamus, and pallidum compared to juveniles, and middle and superior frontal cortex compared to juveniles and young adults (p < 0.05). Future studies should examine the relationship between the changes in gray matter covariation reported here and age-related cognitive decline.