The buffer retainer is supposed to be hollow, the spring fits inside it. The buffer tube should retain it in it's hole, if it came out too far the bolt carrier would hit it and possibly bend or break it. That's what the groove in the bottom rear of the carrier is for, to clear the retainer.
If you have a fixed stock on it, the flange should stop it in the proper spot. The collapsible buffer tube, OTOH, can be screwed in too little and allow the retainer to pop up too far.