I 4-Wheeled in my 1992 Samurai with some friends who had Toyotas and Jeeps on some pretty tough trails and we all did good. We all got stuck in different spots so the differences in them was there to see, but at the end of the day none of us could claim to have a superior vehicle.
I really liked my Sami, and I didn't mod it much. It rode hard though and didn't have any A/C so it wasn't the most comfortable ride off the trail. A stock Tracker is a lot more comfortable ride than a stock Samurai.
I've seen Sidekicks do pretty darn good on some pretty tough trails with just a few mods too. If you're going to do a lift and bigger tires you'll want to gear it down. I put a Rockcrawler Transfer case and 31" tires with 2 inch shackle lift on my Sami and that's all it needed to keep up. I think something similar would turn a Sidekick into a pretty fair trail vehicle. Without the transfer case it wouldn't climb at all, just burn the clutch. With it I could practically idle up steep climbs without any slipping at all.
The main reason I chose the Sami over a Jeep is they're easier to work on. The transmissions and transfer case, and pretty much everything else are all smaller and lighter. The 4cyl engine parts are easy to get at, and overall I'd still say parts will cost you less in the long haul.
The small size of the 2dr models has some advantages on trails that we all learned to appreciate. There were spots I could wiggle through a lot easier than the bigger vehicles I wheeled with, and some places I could get they just couldn't.
Years ago my wife and I had a 1992 Suzuki Sidekick 4dr 4wd that we both loved. It was stock, and I never took it on any 4x4 trails, but I did take it off road on dirt trails that most cars would never get far on and I think they'd make a great trail rig with just minimal mods.
We just bought a 2000 Chevy Tracker 4dr 4wd about a month ago and that was going to be my new trail car but my wife drove it and said she'd be taking it and I could have her old car, a 2000 Merc Cougar. They're no fun at all off road.
So I just bought a 91 Geo/Chevy Tracker 2dr instead and that's my latest "Project Car". I'm not sure if I'll mod it all yet. I'll need to take it on some trails to get a feel for how it does here first but I suspect it will do pretty good as is. It has an auto trans and already has aftermarket wheels and tires that are bigger than stock, so I really do not know what to expect from it. It might need to be geared down too.
I might sell it though, and buy another 4dr instead. I can get one of those here that's still pretty nice and rust free for a pretty decent price and then mod it all I want and end up with a pretty great trail rig.
And that brings me to pretty much where you are and I have to say that's a pretty good way to go. I won't claim to be an expert on off road vehicles but I did build and customized a lot of cars when I was younger, including some "Monster Truck" show cars (but that really doesn't count because they were never driven off road or anywhere). And I've driven a lot of high-end cars (and pushed even more), and I've really come to love these Samurai/Sidekick/Trackers.
Of all the cars I've ever worked on or owned these are my favorite to own and drive.
Here's a link to some photos
of a gathering of Samurais in Disney Oklahoma back in 2001. There were some pretty tough Zuks there.