Pulled crown and stem out (1680)

[dsio]

So, I was adjusting the time, pulled the crown out to hack it, and what do you know, the stem came with it (think this is the right term, the needle like shaft with a square faced end on it attached to the crown) and now when I push that back in, the watch will only move the hands when I turn the crown no matter what position.

Have I done something wrong? I didn't pull excessively hard or anything, though I had just given it a full wind earlier. Is this likely to be a difficult or expensive repair?