Lyric discussion by songmeaningstom 

Dylan may have thought the drug references in this were too blatant. I am suprised so many think it's a religious or other song.

"not my cup of meat" blends the idea of "cup of tea" as personal preference and "meat" as in what primarily drives and sustains you. It's a clever way of saying "I subsist on drugs but I prefer something else, though I know some people live for this."

The idea that "some are building monuments, others jotting notes" probably refers to rolled cigarettes whether joints, laced joints, laced tobacco, etc. Jotting notes would be something narrow and a monument would be obnoxiously large.

Eskimo ~ snow ~ cocaine.

"Trees" likely refers to marijuana as well. The pigeons choose their perches, and when the man with the snow comes along they flock there instead of the trees.

The connection between the actor and film could have a personal meaning to Dylan. Perhaps the actor Anthony Quinn resembled the dealer who inspired the song, or was around the scene either in person or on a poster, and was popularly associated with the archetype of the eskimo at that time and place.

An error occured.