Wednesday left it late as there was just two minutes left on the clock when Darren Potter whipped in a free-kick from 23 yards which swerved wide of the keeper's left hand.
City went in without regular striker Dele Adebola, who has been linked with a summer move to Wednesday.
It meant playing with only one up front in a role that Nicky Maynard found difficult to sustain.
His best move came when he laid off a ball short of the half-hour mark for Lee Johnson - the son of City manager Gary - to strike the home side's goal.
It was a good move started and finished by Johnson in some style as he cut through Sheffield's midfield.
A dull 20 minute start came alive when City's Marvin Elliott found a shot from the edge of the box which flew just wide of Lee Grant's right-hand post.
It sparked a reaction from the visitors which saw Clarke dispossess City's acting captain Liam Fontaine down the right channel only to be robbed in turn when Fontaine chased back to put in a sliding tackle which would have brought a penalty if it had been anything less than 100 percent precise and clean.
Clarke, who was showing no sign of the hamstring injury which had made him a doubtful starter, had a busy afternoon with patchy support.
It would have been better but for an injury to Etienne Esajas right on half-time which saw him stretchered off after a touchline collision with Maynard.