The visitors kicked off attacking the River End goal but found themselves defending a Norwich corner inside the first minute.
Jamie Cureton missed an early chance when the ball bounced off him into the grateful arms of Adriano Basso.
At the other end, Dele Adebola caused problems for the home defence and Liam Fontaine saw a header come back off the crossbar.
Norwich failed to heed this warning and fell behind in the 14th minute when Nicky Maynard's low cross from the right was rifled high into the net by Cole Skuse's right foot from some 16 yards.
Norwich drew level 10 minutes later. Sammy Clingan's right-wing free-kick reached Carl Cort at the far post and when his header came back off the woodwork, Jonathan Grounds side-footed the ball in at the near post with his left boot from five yards.
Maynard came close to scoring with a diving header within seconds of the equaliser.
After 35 minutes, the visitors escaped penalty appeals when Darel Russell's goal-bound shot appeared to be deflected off an outstretched arm for a corner.
David Marshall saved Adebola's header and the big striker fired wide when well-placed.
A minute from half-time, Marshall's weak kick fell to Adebola and as he charged into the penalty area, Doherty appeared to nick the ball off his toe only for Mr D'Urso to brandish a red card and award the penalty which proved to be the winning goal.
Orr sent Marshall the wrong way from the spot with a right-foot strike to the keeper's right.
Norwich did enough attacking in the second half to have earned a draw but referee D'Urso failed to award the penalty for the foul on Drury and Bryan Gunn suffered defeat for the first time in five games since taking charge.