Koji Uehara (1-1) began the home ninth by giving up a single to Alex Rodriguez, then retired Robinson Cano on a deep fly to center. Swisher then lofted the game-winning blast over the wall in left-center to end the game.
The hit made a winner of Joba Chamberlain (3-4), who struck out two in a scoreless top of the ninth.
Brett Gardner added an RBI double for the Yankees, who stemmed a three-game slide and finished their homestand at 7-3. New York moved two games ahead of Tampa Bay in the AL East, with the Rays playing the Red Sox later Wednesday.
The Yankees grabbed a 1-0 lead in the third off O's starter Brad Bergesen. Curtis Granderson led off with a walk, stole second and advanced to third on a grounder by Francisco Cervelli before scoring on a double by Gardner, who was gunned down trying to stretch the hit into a triple.
New York failed to score in the home fifth with two on and one out after Gardner grounded into an inning-ending double play, then Nova ended his outing by working out of a two-on, two-out jam in the sixth when Cesar Izturis lined out to short.
The Bombers also failed despite having runners on the corners and two down in the seventh, as Jim Johnson retired Cervelli on a grounder to third.
Johnson then set down the Yanks in order in the eighth.
For Swisher, it was his third career walkoff home run...It was also his second walk-off hit of the season after his game-winning RBI single on July 16 against Tampa Bay...If the Yankees had lost, it would have marked the first time since June 6-8, 1986 that the Orioles swept a series lasting at least three games in the Bronx...New York has only been swept once this year, by the Rays in a two-game set on May 19-20...Nova allowed six hits and two runs with six strikeouts and two walks over six innings, and Bergesen scattered four hits and a run while fanning five in 6 1/3 frames.