In disordered organic semiconductors, excited states and charges move by hopping in an inhomogeneously broadened density of states, thereby relaxing energetically (“spectral diffusion”). At low temperatures, transport can become kinetically frustrated and consequently dispersive. Experimentally, this is observed predominantly for triplet excitations and charges, and has not been reported for singlet excitations. We have addressed the origin of this phenomenon by simulating the temperature dependent spectral diffusion using a lattice Monte Carlo approach with either Miller–Abrahams or Förster type transfer rates. Our simulations are in agreement with recent fluorescence and phosphorescence experimental results. We show that frustrated and thus dispersive diffusion appears when the number of available hopping sites is limited. This is frequently the case for triplets that transfer by a short-range interaction, yet may also occur for singlets in restricted geometries or dilute systems. Frustration is lifted when more hopping sites become available, e.g., for triplets as a result of an increased conjugation in some amorphous polymer films.