The Blades had taken a two goal first half lead but City levelled the scores in the second half.
However, a Darius Henderson stoppage time winner meant the visitors left Ashton Gate with all three points much to the frustration of the City boss.
"We have conceded a lot of late goals this season and this was a real gut-wrencher," Johnson said after the match.
"I'll be looking hard at the match video and the ones from our previous games to see if there is a common denominator about the late goals.
"Sheffield's first two were down to individual errors that you can't legislate for. I thought overall it was one of our better home displays.
"We now have another home game against Ipswich Town coming up and need to get our form back on track.
"The lads deserve credit for working really hard in the second half, but in the end it all counted for nothing."