Context. Comparisons of optical positions derived from the Gaia mission and radio positions measured by very long baseline interferometry (VLBI) probe the structure of active galactic nuclei (AGN) on the milliarcsecond scale. So far, these comparisons have focused on using the S ∕ X -band (2/8 GHz) radio positions, but did not take advantage of the VLBI positions that exist at higher radio frequencies, namely at K -band (24 GHz) and X ∕ Ka -band (8/32 GHz). Aims. We extend previous works by considering two additional radio frequencies ( K -band and X ∕ Ka -band) with the aim to study the frequency dependence of the source positions and its potential connection with the physical properties of the underlying AGN. Methods. We compared the absolute source positions measured at four different wavelengths, that is, the optical position from the Gaia Early Data Release 3 (EDR3) and the radio positions at the S ∕ X -, K -, and X ∕ Ka -band, as available from the third realization of the International Celestial Reference Frame (ICRF3), for 512 common sources. We first aligned the three ICRF3 individual catalogs to the Gaia EDR3 frame and compared the optical-to-radio offsets before and after the alignment. Then we studied the correlation of optical-to-radio offsets with the observing (radio) frequency, source morphology, magnitude, redshift, and source type. Results. The deviation among optical-to-radio offsets determined in the different radio bands is less than 0.5 mas, but there is statistical evidence that the optical-to-radio offset is smaller at K -band compared to S ∕ X -band for sources showing extended structures. The optical-to-radio offset was found to statistically correlate with the structure index. Large optical-to-radio offsets appear to favor faint sources, but are well explained by positional uncertainty, which is also larger for these sources. We did not detect any statistically significant correlation between the optical-to-radio offset and the redshift. Conclusions. The radio source structure appears to be a major cause for the radio-to-optical offset. For the alignment of the Gaia celestial reference frame, the S ∕ X -band frame remains the preferred choice at present.< Réduire