Residual diagnostics is very important for frequentist normal regression modelling. We can check the overall goodness-of-fit (GOF) of a fitted model by checking the normality of residuals with QQ plots and statistical tests. However, residual diagnostic tools are not available for Bayesian models. In this talk, we propose a method to define residuals for checking Bayesian models, which is called Z-residual. The Z-residual is transformed from the cross-validatory randomized predictive p-values (RSP) with the normal quantile function. We show that the RSP has a uniform distribution on $(0,1)$ when the distributions for the likelihood and the prior are correctly specified. Due to the uniformity of the RSPs, Z-residuals are normally distributed under the true model. Therefore, we can use the Z-residual to conduct residual diagnostics for Bayesian models as for normal regression. Applying Z-residual diagnostics to a Poisson disease mapping model with spatial effects, we demonstrate that the graphical diagnostics based on Z-residuals can effectively identify the mis-specification in the distributional family for the response variable. We also investigate the sizes and powers of statistical tests based on the Z-residual with large-scale simulation studies.