In Drosophila, the function of the Polycomb group genes (PcGs) and their target sequences (Polycomb response elements (PREs)) is to convey mitotic heritability of transcription programmes—in particular, gene silencing. As part of the mechanisms involved, PREs are thought to mediate this transcriptional memory function by building up higher-order structures in the nucleus. To address this question, we analysed in vivo the three-dimensional structure of the homeotic locus bithorax complex (BX-C) by combining chromosome conformation capture (3C) with fluorescent in situ hybridization (FISH) and FISH immunostaining (FISH-I) analysis. We found that, in the repressed state, all major elements that have been shown to bind PcG proteins, including PREs and core promoters, interact at a distance, giving rise to a topologically complex structure. We show that this structure is important for epigenetic silencing of the BX-C, as we find that major changes in higher-order structures must occur to stably maintain alternative transcription states, whereas histone modification and reduced levels of PcG proteins determine an epigenetic switch that is only partially heritable.