Markov Random Fields (MRFs) are a popular tool in many computer vision problems and faithfully model a broad range of local dependencies. However, rooted in the Hammersley-Clifford theorem, they face serious difficulties in enforcing the global coherence of the solutions without using too high order cliques that reduce the computational effectiveness of the inference phase. Having this problem in mind, we describe a multi-layered (hierarchical) architecture for MRFs that is based exclusively in pairwise connections and typically produces globally coherent solutions, with 1) one layer working at the local (pixel) level, modelling the interactions between adjacent image patches; and 2) a complementary layer working at the object (hypothesis) level pushing toward globally consistent solutions. During optimization, both layers interact into an equilibrium state, that not only segments the data, but also classifies it. The proposed MRF architecture is particularly suitable for problems that deal with biological data (e.g., biometrics), where the reasonability of the solutions can be objectively measured. As test case, we considered the problem of hair / facial hair segmentation and labelling, which are soft biometric labels useful for human recognition in-the-wild. We observed performance levels close to the state-of-the-art at a much lower computational cost, both in the segmentation and classification (labelling) tasks