Damage identification techniques are used to assess the structural condition and further ensure structural safety and reliability. The proposed approach for damage identification is based on integrating parameterized physics-based models with multiple datasets collected under various operational and environmental conditions. Damage is identified by tracing the changes in the structural model parameters. Due to model and measurement errors, uncertainties inevitably exist during the process of identifying structural damage. Model error can cause variabilities in the identified condition of the structure when multiple datasets are processed, persisting even under the same operational and environmental conditions. Such variabilities undermine the effectiveness of damage identification techniques, often leading to inaccurate results or false alarms. To this end, this work presents a hierarchical Bayesian learning framework for identifying the location and size of structural damage in the presence of variabilities arising from model error. It is assumed that a sufficiently large number of datasets is collected from the structure at its healthy and its damage condition. A physics- based digital twin is learned and a probabilistic model for the damage index is introduced based on posterior distribution of the model parameters, accounting for the aforementioned variabilities. The estimation is based on sampling algorithms as well as computationally efficient asymptotic approximation tools introduced to considerably improve the computational burden of the proposed framework. The performance of the framework is illustrated using multiple data sets collected from the healthy and damage states of an example dynamical structure. Results indicate that the proposed framework provide robustness of structural damage identification to uncertainties. The framework has great potential to be applied to various infrastructure components, including bridges, buildings, offshore structures and wind turbines.