I'm in the camp they mistakenly over assumed the weight of the sex in the perfect happiness. But how do you test that theory without running a rather high risk? Angel did end up testing that theory with Darla. Both of them appeared to believe that Angel having sex with her had a high likelihood of separating him from his soul. And now that he knows he could lose his soul, now that he's got Connor to worry about and now that he lives with the losses of people he cared deeply about, it strikes me as really unlikely he could lose his soul, even if it was sex with Buffy. It would have to be a rather remarkable confluence of events.