You're either lucky or have terrible bases :). Raiding players is one of the best ways to get large quantities of resources if it's done properly. The other players that CAN see you may not find you a worthwhile target.
Moles and raiders are almost as crap as Wasps. Their point-blank range gets them into range of heavy defenses like Mortars and defense bunkers where they will get pulverized long before they can make back their production costs. Avoid them at all costs.
Don't play this much anymore but I do remember my defense layout. I went with a "line" layout with the line stretching between the corners (so straight "up-down" or "left-right" on the map given its default view).
The main base was at the very center of the map and all of my defensive structures were lined up as the "spine" of the line, with no defensive structure type clustered together in one spot; everything was spread out. Around the defense structures I put my resource structures to act like protection. I also included my gates so that if I had a large army built up waiting in them they would be protected and I wouldn't potentially lose that large army in an attack. The research and unit production buildings I left completely outside the line; losing them does very little and I could keep the main defenses better concentrated. Finally, I was able to surround the entire line completely with walls with no gaps, so attacking troops including S-trikes would be forced to shoot through them.
With this layout, no matter where anyone attacked from they would be in range of multiple heavy defense structures a.k.a. defense bunkers and Mortars. I saw that layout sometimes survive four attacks in a row from high level players and still have multiple buildings intact.
Well, a level 4 starbase is quite low level; using these is a bit of cost overkill but yes they would be quite effective. Higher than that where you start running into defense bunkers with the Colossus, these can really get into trouble.
They are up to a point. Once you start hitting/getting hit by bases with Colossi in them these die very fast if you aren't careful. However, as a mixed force with Colossi they can work really well; try to empty the bunkers and at least one missle turret first then send these in to take out mortars, or at least as many turrets as they can before they fall.