We investigate methods of estimating a block diagonal structure of Gaussian covariance matrices. The space of all possible partitions of variables is large and infeasible to search in large dimensions. Through hierarchical clustering we are able to restrict the search space and obtain a sequence of nested partitions. We propose a Ward procedure where each partition is chosen based on the maximal log-likelihood. We compare the performance of this method to single, average, and complete linkage hierarchical clustering based on the sample correlation matrix in a simulation study. We then extend this methodology to Gaussian mixture models and demonstrate its performance through a simulation study and data analysis.