Sounds like the starter to me as well, I've driven chevys before for months having to bang on the starter before they'd crank over. And that key pulling out thing is very common.
I have a '79 RS that's in the middle of a frame off, and has been basically for the last couple of decades
It was my first car, so I have a major soft spot for second gen soft nose camaros, despite their um... limitations (160 hp 350??.. yikes).
