In the uncertainty quantification and structural reliability evaluation, epistemic uncertainty usually exists simultaneously with the aleatory uncertainty. To characterize these two types of uncertainty is crucial for decision making in structural design. Though extensive investigations have been conducted, resulting in various approaches, including, e.g., the fuzzy analysis method, the imprecise probability and empirically based probabilistic method, etc., a compatible framework with efficient implementation is still in need. Actually, due to limited available data, the distribution parameters (e.g. mean value and standard deviation) carry epistemic uncertainty, thus the basic random variables should be characterized by a family of probability distributions, rather than an uniquely specified probability distribution. This set of distribution parameters can be determined by the bootstrap method. Such problem is addressed in the present paper. Moreover, to improve the computational efficiency, a newly proposed method called PDEM-COM is adopted, without compromising numerical accuracy. Numerical applications are illustrated to indicate the feasibility of PDEM-COM framework for quantification of epistemic uncertainty. Moreover, this basic idea can also be extended to quantification of epistemic uncertainty due to other sources.