While numerous methods have been developed to analyze high-dimensional data, there is comparatively little work on model diagnostics for assessing the validity of results and selecting amongst different proposed models for a given type of problem. Methodology routinely used for problems such as two-sample testing, variable-selection, and graphical model inference for high-dimensional biological data can be particularly sensitive to departures from model assumptions, but there is often limited guidance and available techniques for detecting these departures. To address these issues, we develop some Bayesian and Frequentist goodness-of-fit and diagnostic approaches based on penalized eigenvalue estimation and some new results on inter-point distance properties. This methodology is then used to analyze several genomic datasets.