Given the importance of quantum reference frames (QRFs) to both quantum and gravitational physics, it is pertinent to develop a systematic method for switching between the descriptions of physics relative to different choices of QRFs, which is valid in both fields. Here we continue with such a unifying approach, begun in [Quantum 4, 225 (2020)], whose key ingredient is a symmetry principle, which enforces physics to be relational. Thanks to gauge related redundancies, this leads to a perspective-neutral structure which contains all frame choices at once and via which frame perspectives can be consistently switched. Formulated in the language of constrained systems, the perspective-neutral structure is the constraint surface classically and the gauge invariant Hilbert space in the Dirac quantized theory. By contrast, a perspective relative to a specific frame corresponds to a gauge choice and the associated reduced phase and Hilbert space. QRF changes thus amount to a gauge transformation. We show that they take the form of `quantum coordinate changes'. We illustrate this in a general mechanical model, namely the relational N -body problem in 3D space with rotational and translational symmetry. This model is especially interesting because it features the Gribov problem so that globally valid gauge fixing conditions, and hence relational frame perspectives, are absent. The constraint surface is topologically non-trivial and foliated by 3-, 5- and 6-dimensional gauge orbits, where the lower dimensional orbits are a set of measure zero. The N -body problem also does not admit globally valid canonically conjugate pairs of Dirac observables. These challenges notwithstanding, we exhibit how one can construct the QRF transformations for the 3-body problem. Our construction also sheds new light on the generic inequivalence of Dirac and reduced quantization through its interplay with QRF perspectives.