This work highlights methodological challenges in isolating a vaccine's effect on progression to severe disease after infection. Vaccines can reduce an individual's risk of infection and their risk of progression to disease given infection. The latter effect is less commonly estimated but is relevant for risk communication and vaccine impact modeling. Using a motivating example from the COVID-19 literature, we note how vaccine effectiveness against progression can appear to increase over time in settings where true biological strengthening is unlikely. We use mathematical modeling to demonstrate how this phenomenon can occur when there is an underlying vulnerable subpopulation with poor vaccine response against infection and progression. We describe a modeling framework to link underlying immunology and post-vaccination outcomes that we use to further examine this problem.