We present numerical simulations of the impact of laser beam wavefront aberrations in cold atom interferometers. We demonstrate that to reach accuracy at the mrad level, simulations cannot be based on a description of the retroreflection optics only with low-order Zernike polynomials, as the results will then depend on the decomposition order and the decomposition technique chosen. Moreover, simulations with high-order Zernike polynomials or equivalently high spatial frequency components require the propagation of aberrations to be taken into account, rather than adding them to the ideally propagated beam. Finally, we examine the impact of the parameters of the atomic source and show that the use of delta-kicked atomic cloud would efficiently mitigate the impact of this systematic effect.