Lyric discussion by thriggle 

I agree with Asdfaeou's assessment.

It's also worth noting that Andrew Bird is playing off the old children's rhyme:

Rubin, Rubin, I've been thinkin' What on earth have you been drinkin' Looks like water, tastes like wine Oh my gosh, it's turpentine

(Different variations of the rhyme exist, of course)

An error occured.