Ultrasonic scattering at grain boundaries in polycrystalline materials causes sound velocity dispersion and the attenuation of sound waves in propagation direction. Both effects as well as the amplitudes of the scattered waves and their directional dependence can be used for materials characterization. Scattering waves emanating from defects are superimposed by grain scattering (“grain noise”), which influences considerably the detectability and measurability of defects by ultrasonic techniques. As a consequence, scattering at the microstructures of materials has to be included into ultrasonic nondestructive testing evaluations and simulations. Thus a convenient model describing ultrasonic scattering in polycrystals is required. Directional dependent ultrasonic grain scattering coefficients have been calculated using the first order Born approximation of the waves scattered at a single grain within an isotropic matrix, the ensemble average of the material. Furthermore, the single scatterer far-field is used, i.e. the approach does not comprise close-packed scatterers, which actually are existent in polycrystalline materials. This approximation is valid only for low scattering and in the Rayleigh regime (low frequency limit). For materials with a high single crystal anisotropy factor, as are e.g. most steels, the resulting scattering amplitudes are too low. In this contribution, ultrasonic grain scattering coefficients are calculated from the ensemble averaged equation of motion in the Born and in the Keller approximation of second order. The exploitation of the underlying analytical formalism for sound velocity dispersion and attenuation calculations already described in the literature has shown the power of this completely different approach which is also for large scattering and for higher frequencies. Here, the formalism is used for the calculation of the so-called grain noise, the scattering waves in single phase polycrystals.