In this paper windowed DFT based channel estimation combined with ICI self-canellation scheme is proposed to combat against ICI caused by frequency offset in underwater acoustic multipath fading channel with non-sample-spaced path time delays. Simulation results prove that this proposed scheme can reduce aliasing error and ICI effects effectively and achieve superior performance.