The former Roma man struck eleven minutes from time to end Lazio's unbeaten start and send Juventus clear at the top of Serie A. It was classic smash and grab stuff from the Marassi outfit, and extra special for Borriello who is now off the mark in his second stint with the Grifone.
Lazio did everything but score. Chance after chance went begging as the Eagles couldn't find a way past custodian Sebastian Frey.
Lazio saw an early effort cleared off the line. Kozak sprung the offside trap, and rounded Frey, but Michele Canini cleared the ball before it went into the net.
Borriello saw a looping header land on the roof of the net, while Frey could only watch as a Hernanes screamer skimmed the upright.
Senad Lulic pulled back for a Zarate volley which just went over, while Frey frustrated Antonio Candreva with a couple of stops.
Mario Sampirisi almost scored for the visitors after reacting quickest to a free-kick. The Lazio defence was caught cold, but the defender's effort went into the side netting.
Hernanes curled a shot against the crossbar at the start of the second half, before Candreva was denied again.
At the other end, Federico Marchetti partially cleared an Andreas Granqvist header, while Borriello's follow-up was charged down. Borriello was then denied a penalty after a tussle with Lionel Scaloni, but it was six of one, half a dozen of another.
Petkovic rung the changes with three substitutions - one of which was Klose for the final 15 minutes. The German almost made an immediate impact, firing off a shot which beat Frey, but trickled past the wrong side of the post.
The visitors then scored against the run of play. Borriello ran on to Luca Antonelli's flick-on, got behind Giuseppe Biava and fired a shot past Marchetti to silence the Olimpico crowd. To make matters worse for Biava, he went down with cramp, but couldn't go off as Lazio had used up all their changes.
Emiliano Moretti prevented substitute Stefano Mauri from meeting a Lulic cross at the far post before leaving the field with a back problem. Klose did put the ball in the visitors net, but the effort was disallowed because the German was one of three Lazio players clearly stood in an offside position.