It's Only Love

Heard this in a pub and these Beatles lyrics keep rattling around in my head in London tonight --

I get high when I see you go by, my oh my
When you sigh, my, my inside just flies, butterflies
Why am I so shy when I'm beside you?

It's only love and that is all, why should I feel the way I do?
It's only love and that is all but it's so hard loving you

Was it Rubber Soul or Revolver? I can never remember. Hell, I was only 9 years old when it was new. Wait, isn't that supposed to be "easy loving you" in that last line?