The Hartree-Fock ground state of the Homogeneous Electron Gas is never translation invariant, even at high densities. As proved by Overhauser, the (paramagnetic) free Fermi Gas is always unstable under the formation of spin or charge density waves. We give here an explicit bound on the energy gain due to the breaking of translational symmetry, which is exponentially small at high density. This justifies a posteriori the use of the non-interacting Fermi Gas as a reference state in the large-density expansion of the correlation energy of the Homogeneous Electron Gas.