Objective: Extensive research using water-diffusion MRI reported age-related modifications of cerebral White Matter (WM). Moreover, water-diffusion parameter modifications have been frequently associated with cognitive performances in the elderly sample, reinforcing the idea of aging inducing microstructural disconnection of the brain which in turn impacts cognition. However, only few studies really assessed over-time modifications of these parameters and their relationship with episodic memory outcome of elderly. Materials and Methods: One-hundred and thirty elderly subjects without dementia (74.1 ± 4.1 years; 47% female) were included in this study. Diffusion tensor imaging (DTI) was performed at two-time points (3.49 ± 0.68 years apart), allowing the assessment of changes in water-diffusion parameters over time using a specific longitudinal pipeline. White matter hyperintensity (WMH) burden and gray matter (GM) atrophy were also measured on FLAIR and T1-weighted sequences collected during these two MRI sessions. Free and cued verbal recall scores assessed at the last follow-up of the cohort were used as episodic memory outcome. Changes in water-diffusion parameters over time were included in serial linear regression models to predict retrieval or storage ability of elderly. Results: GM atrophy and an increase in mean diffusivity (MD) and WMH load between the two-time points were observed. The increase in MD was significantly correlated with WMH load and the different memory scores. In models accounting for the baseline cognitive score, GM atrophy, or WMH load, MD changes still significantly predict free verbal recall, and not total verbal recall, suggesting the specific association with the retrieval deficit in healthy aging. Conclusion: In elderly, microstructural WM changes are good predictors of lower free verbal recall performances. Moreover, this contribution is not only driven by WMH load increase. This last observation is in line with studies reporting early water-diffusion modification in WM tissue during aging, resulting lately in the appearance of WMH on conventional MRI.< Réduire