Power spectra of de-projected images of late-type galaxies in gas and/or dust emission are very useful diagnostics of the dynamics and stability of their interstellar medium. Previous studies have shown that the power spectra can be approximated as two power-laws, a shallow one at large scales (larger than 500 pc) and a steeper one at small scales, with the break between the two corresponding to the line-of-sight thickness of the galaxy disk. We present a thorough analysis of the power spectra of the dust and gas emission at several wavelengths in the nearby galaxy M33. In particular, we use the recently obtained images at five wavelengths by PACS and SPIRE onboard Herschel. The large dynamical range (2-3 dex in scale) of most images allow us to determine clearly the change in slopes from -1.5 to -4, with some variations with wavelength. The break scale is increasing with wavelength, from 100 pc at 24 and 100micron to 350 pc at 500micron, suggesting that the cool dust lies in a thicker disk than the warm dust, may be due to star formation more confined to the plane. The slope at small scale tends to be steeper at longer wavelength, meaning that the warmer dust is more concentrated in clumps. Numerical simulations of an isolated late-type galaxy, rich in gas and with no bulge, like M33, are carried out, in order to better interpret these observed results. Varying the star formation and feedback parameters, it is possible to obtain a range of power-spectra, with two power-law slopes and breaks, which nicely bracket the data. The small-scale power-law is indeed reflecting the 3D behaviour of the gas layer, steepening strongly while the feedback smoothes the structures, by increasing the gas turbulence. M33 appears to correspond to a fiducial model with an SFR of $\sim$ 0.7 Mo/yr, with 10% supernovae energy coupled to the gas kinematics.Read less <