Actually saltwater is not that much more difficult. Coral, sea-cucumbers, and things like that are very expsensive and difficult, but to own a fish only tank is not that bad. I use the same sponge filter in my saltwater as in my freashwater. And you dont need all the live-rock and live-sand and crushed coral that they tell you that you need. They only tell you that so that it all makes a mess in your tank, then they sell you all the stuff to clean up the mess. The thing that kills you on saltwater is that fish are very expensive, so if you do lose one it hurts. But they are worth it because you don't have to tease the color out of them; they are colorful all the time. And some of the predators are just the coolest looking things you will ever see.
have to differ with you there. yes salt-water isn't "that hard", but it can still present difficulties (as can keeping any tank)
however, live rock and live sands do have a purpose, and can be very effective biofilters. and the clean up crew generally go after detrius and algae that accumulates.
Fish Only (FO) is one of the easiest marine setups you can get, you just need to ensure that the choice of fish is suitable for the tank and compatible with each other.
reef tanks are a lot harder in the extent that you have to provide the corals with what they need (high light(for most sp) good water movement) as well as keeping the water a lot cleaner than needed for a FO tank.
IMO expense is the biggest thing with marine tanks, for a simple FO its comparable to a freshwater tank (might need better filtration (wet/dry towers for better oxygenation) ) and obviously running costs are higher with the requirement to buy marine salt.
reef tanks though can get extortionately higher, with metal halides, calcium reactors, protein skimmers. etc etc
note I've made generalizations, there are some situations that are easier or harder than I might have made out