Lyric discussion by John_Cale 

Yes the person in the song (we can safely assume Lou writes from personal experiene) is going to the corner of Lexington and 125th to score heroin. You can get the subway up there (4,5,6 I believe) which is part of its appeal as a drug dealing area. It's uptown (as the 2nd verse points out) in Harlem hence the African-American men questioning why a white man would be so far uptown and obviously not a resident. Eventually "the man" (i.e. the dealer) shows and he shoots up at which point in spite of his woman yelling at him he doesn't care. At least until he comes down again.

An error occured.