Spatially varying distortions in optical elements-for instance prisms and progressive power lenses-modulate the visual world disparately in different visual areas. Saccadic eye movements in such a complexly distorted environment thereby continuously alter the retinal location of the distortions. Yet the visual system achieves perceptual constancy by compensating for distortions irrespective of their retinal relocations at different fixations. Here, we assessed whether the visual system retains its plasticity to distortions across saccades to attain stability. Specifically, we tapped into reference frames of geometric skew-adaptation aftereffects to evaluate the transfer of retinotopic and spatiotopic distortion information across saccades. Adaptation to skew distortion of natural-image content was tested at retinotopic and spatiotopic locations after a saccade was executed between adaptation and test phases. The skew-adaptation information was partially transferred to a new fixation after a saccade. Significant adaptation aftereffects were obtained at both retinotopic and spatiotopic locations. Conceivably, spatiotopic information was used to counterbalance the saccadic retinal shifts of the distortions. Therefore, distortion processing in a natural visual world does not start anew at each fixation; rather, retinotopic and spatiotopic skew information acquired at previous fixations are preserved to mediate stable perception during eye movements.