Well, some apps fail to close and just pop up sometimes either because they're being resumed by some system service (especially if it's one of those bloatware that comes with the phone), or if you have related services activated. Using that power control thingy, if I deactivate the automatic GPS it doesn't only stop tha a-GPS consumption, but the Maps and My Place services are also gone (moreorless, but still...). And also the permanent sync, that's what constantly recalls some mail programs and weather and stuff.
And yes, the app killer IS an app, so it drains battery too.
But it's written everywhere that since Froyo, having an App Killer is pointless. Why do people still use that stuff then?

I presonally prefer Norton Utilities. It has so many features built-in! (an app killer too)
No, I'm not advertising it, it's just plain great!