Caves form in limestone formations. The only problem with getting warmer as you go deeper is that, absent warm mineral springs, it takes a few thousand feet of depth to gain temperature. A cave thousands of feet deep is going to be ultra rare due to the nature of cave formation.
Even if one were available, stocking supplies, managing sanitary facilities, and simply navigating to depth will be problematic. Again, due to the nature of cave formation, the trek into the depths will include vast rooms as well as crawlspaces barely large enough for a determined caver to pass.
"Normal" caves maintain somewhere in the vicinity of constant 50-60 degree farenheit temperatures. This is cold enough that, as I stated before, the most common cave rescue operation involves evacuating a hypothermia victim. There is no sun, warm breeze or anything except tons of rock at that temperature, so it is difficult to raise the temperature around you without insulation to ward off the immense heatsink that's all around you. Insulation and a heat source are vital unless you plan on staying right at the cave entrance.
I have a daughter. I tell her, "911 is what you dial after you're raped. 1911 is what you should have before they try."