Two trends play an increasing role in shaping the long-term prospects of children across the United States: a growing STEM (science, technology, engineering, and mathematics)-based economy and rising income segregation. Together, they suggest that the future well-being of U.S. children may be based increasingly on the contemporary interplay of mathematical learning and neighborhood environments. Drawing on data from the Panel Study of Income Dynamics and a value-added design, this article provides a rigorous test of the hypothesis that exposure to high-poverty neighborhoods exerts a depressive effect on children’s mathematics achievement growth beyond that associated with individual-, family-, and school-level characteristics. Results indicate that exposure to higher-poverty neighborhoods reduces children’s mathematical growth, a treatment effect roughly equivalent to missing three quarters of a year of schooling over the 5-year treatment period. A formal sensitivity analysis suggests that these effects are remarkably robust to unobserved residual confounding.