If external coordinate reference frame is _not_ established early in life, you do not develop it and everything in your world is relative to you.
“Developmental Aspects: Blind Individuals
Even though remapping does occur in absence of vision, for instance when locating tactile stimuli in the dark or with a blindfold (Kóbor et al., 2006; Schicke and Röder, 2006), several studies suggest that tactile remapping is closely related to the development of the visual system during ontogeny. For instance, congenitally blind participants were unaffected by crossing the hands when performing a TOJ (Röder et al., 2004, see Figure 4). Strikingly, people who had turned blind later in life performed just like the sighted, and showed a marked crossing effect. Furthermore, a man born with bilateral cataracts and, thus, functionally blind, and whose vision was surgically restored at age 2, did not exhibit a crossing effect (Ley et al., 2013). Even more, this man did use external coordinates for the representation of touch in a task that involved bimodal, visual, and tactile, stimulation. These results suggest a pivotal role for the visual system during early life for the development of coordinate transformations in touch: if vision is available after birth, then the default use of an external reference frame is established and remains intact, even if vision is lost at a later point in time. In contrast, when vision is not available after birth, then the tactile system does not seem to integrate an external reference frame as a default source of spatial information, even if vision becomes available later. At least if vision is restored early on, then the use of external coordinates in touch can be established for specific situations, presumably predominately those involving the integration of touch with vision.”