If you use Sched 40 you don't have to seal it in permanently. You can install on one end of the container a "screw cap" or a "screw-plug." This way you can open it anytime you want. These fittings are a bit expensive on the large sizes, but you can make your containers more than just a one shot thing (no pun intended). Do some research on plumbing size supply sites and post the links please.
And as for the welding--you have to purchase the welding rod to match the material that you are using. It's usually sold by the pound. Do a search for plastic welding rod. I didn't know that harbor freight had them--my tool is a bit more pricey.
Also... you can, once end-caps are installed on a PVC container weld a bead around the seam as extra insurance. However you have to let the glue dry for a couple days at least.
Hey let's see some photos of your completed projects!!!
long-term, moisture-proof storage is beginning to become a concern for me at this point.
If you were to store firearms--I'd oil them up thouroughly and slide one of those no-rust plastic socks over them. Also before you seal the container, you should aim a hair dryer into the container for a while to drive out all the moisture. Then at the last moment, toss in your desicant, and seal the puppy up. Temp changes will cause moisture to condense within the container. This will effect metal objects greatly, but other things will be just fine.