I get very few visitors up my way. My neighbors (the few that there are) are similarly isolated. I live in the high country and any "roaming gangs" are more likely to meet their demise long before they find their way into the hinterlands where I am at. That said, I have multiple BOLs and cached supplies. As for defense - there are several possibilities.
First, I have family and friends who are with the program. Some are more determined than others and I even have some Mormon family and Friends who are big into prepping and defense (it's part of their religion after all). Having like-minded individuals band together is a big plus.
Second, We have alternative BOL sites across four states now. Depends on your "community" and ours have been at this for years. Some are better prepped than others and some have more desirable skills depending on th situation (i.e. a doctor, physician's assistant, a ER nurse, two mechanics, two current military, etc.). Our extended community probably number 60 individuals.
Third, there are cached supplies along routes to each BOL. Some known only to each group leader. Most keep in a database on encrypted PGP software.
Fourth, always have a defensive perimeter defined and set up early warning systems (cameras, motion detectors, man traps, etc.).
As it is I am currently working on setting up one site and may be buying another site with water sources on wooded acreage in the high country. The price is a bit steep but may see if it's still available in a few months when I have the cash accumulated (always buy raw land with cash and if possible held in a family trust or investment trust for liability and tax purposes).
There's much more but these are the very bottom line basics. The most difficult part for most people of course is finding the right sort of individuals. Many of us began around the mid to late 1970s. The kicker for me was the book by Howard J. Ruff - How to Prosper During the Coming Bad Years (1979). Made a lot of sense and helped me and my late wife make it through some difficult times and medical expenses during her illness.
As for anything else - get out of debt and stay out of debt, accumulate physical silver and gold as "portfolio insurance", stock a multiyear supply of nonperishable foods and basic necessties (and of course that includes firearms and ammo for defense and hunting needs).
When you're born you get a ticket to the freak show. When you're born in America , you get a front row seat. - George Carlin