Pressure-driven coarsening triggers bubble rearrangements in liquid foams. Our experiments show that changing the continuous phase rheology can alter these internal bubble dynamics without influencing the coarsening kinetics. Through bubble tracking, we find that increasing the matrix yield stress permits bubble growth without stress relaxation via neighbor-switching events, promoting more spatially homogeneous rearrangements and decoupling bubble growth from mobility. This eventually leads to a structural change that directly impacts the foam mechanical and stability properties, essential for applications in various technological and industrial contexts.