In this work we quantify the uncertainty over Power Spectral Density estimation of stochastic processes based on realizations with gapped missing data. For the purpose of imputation, a fully-connected neural network architecture that works in an autoregressive manner is firstly constructed to probabilistically capture the temporal patterns in the time series data. Particularly, under the Bayesian scheme, uncertainties with respect to the parameters of the neural network model (i.e. weights) are introduced by multivariate Gaussian distribution. During training, the posteriors are learnt through variational inference approach. As a result, the missing gaps can be recursively imputed via our neural network in each realization, and thanks to the probabilistic merit of Bayesian inference, an ensemble of reconstructed realizations can then be obtained. Further, by resorting to a Fourier-based spectral estimation method, a probabilistic power spectrum could be derived, with each frequency component represented by a probability distribution.