Why its it so hard to move on

I've hated my job for a long time. I've carried the workload while lazy people sit back and do nothing but say "I don't know I don't know". I make less money than everyone else. New hires come in making more than me. Everyone is on my back constantly for everything, its neverending BS. I've gone to interviews at other places and made excuses to myself why I should stay put. For the most part, I've stayed put because other places won't even match what I make in the looney bin. I've been grinning and bearing that place for years. Why then, when I finally get an interview set up at another company for a job that actually pays more money, a job that I'm actually excited about why, why why to people put me down? Why can't I get any support? Why do I have to be afraid to move on? Is it wrong to want to take a chance and start over again?