Initially, the obsessive thoughts. I had the ability to focus on other things and temporarily not think about smoking. However, if I wasn't tuned in completely to something else, the smoking thoughts were always at or near the surface.
The only long-term issues I dealt with, which I would classify more as annoying rather than hard, were: the desire to smoke after finishing a job or task and the inability to fully identify as a non-smoker. I was several months into my quit before the thoughts of wanting a cigarette upon completion of a job began to cease. The identity issue was, I suppose, a combination of a lingering addiction and a bit of self-doubt. Even as I put more milestones behind me, it often felt like I was just taking a break from smoking before I inevitably started smoking again.
In hindsight, the "hard" parts weren't really all that hard. And considering what I've gotten in return for my efforts, quitting smoking is one of the best bargains of my life.