Progressive shortening of telomeres ultimately causes replicative senescence and is linked with aging and tumor suppression. Studying the intricate link between telomere shortening and senescence at the molecular level and its population-scale effects over time is challenging with current approaches but crucial for understanding behavior at the organ or tissue level. In this study, we developed a mathematical model for telomere shortening and the onset of replicative senescence using data from Saccharomyces cerevisiae without telomerase. Our model tracks individual cell states, their telomere length dynamics, and lifespan over time, revealing selection forces within a population. We discovered that both cell genealogy and global telomere length distribution are key to determine the population proliferation capacity. We also discovered that cell growth defects unrelated to telomeres also affect subsequent proliferation and may act as confounding variables in replicative senescence assays. Overall, while there is a deterministic limit for the shortest telomere length, the stochastic occurrence of non-terminal arrests drive cells into a totally different regime, which may promote genome instability and senescence escape. Our results offer a comprehensive framework for investigating the implications of telomere length on human diseases.