Just to add to Don's reply, the one on 8th and 31st is on a side street, between the Pennsylvania Hotel and a Sbarro's, right across the street from Madison Square Garden and the Borders. I walk by the line every day on my way to work.
The Megabus terminal in New York City is located at the corner of West 31st Street and 8th Avenue, right outside of Penn Station. The link below is the location of the bus stop.
Megabus uses separate locations in New York for arrivals and departures. MEGABUS.COM departures are located adjacent to Penn Station, on the north side of West 31st Street at the northeast corner of 31st Street and 8th Avenue. MEGABUS.COM arrivals are located at the MTA bus shelter on the west side of 7th Avenue, just south of 28th Street. PLEASE NOTE that Academy services to and from Atlantic City depart and arrive at Port Authority Bus Station, Gate 1.