I was reading Vonnegut's Timequake and he said that one of the reasons given for the fall of Rome was that they used lead pipes and lead poisoning can make you stupid and lazy.
As good a reason as any.
By this time, the Western Roman army was more or less composed mainly of foreign mercenaries under Roman commanders due to the Roman citizenry's hesitance to serve. The fact that the Western Empire was flat broke only exacerbated things. To quote RoboCop from an episode of Engineering an Empire, "lack of a trained army and subsequently no defense." At least, that's (the short version of) my view on it.