One night after he set the Braves franchise record for career saves, Craig Kimbrel blew his third of the season in 19 opportunities.
The Diamondbacks were 0 for 12 with runners in scoring position before Aaron Hill's pinch-hit double off Kimbrel down the right field line scored Ender Inciarte from second to tie it at 2-2.
Jason Heyward homered with two outs in the 10th off Addison Reed to put the Braves up 3-2, but Miguel Montero tied it again with an opposite-field, one-out shot off Anthony Varvaro in Arizona's half of the inning.
Ross singled with one out off David Carpenter (4-1), then Didi Gregorius walked to bring up Parra, whose soft fly ball fell in front of center fielder B.J. Upton.
Randall Delgado (1-1) struck out the side after a leadoff walk in the 11th to get the victory.
It was just the 18th blown save in 173 career opportunities for Kimbrel, still the third-highest percentage in baseball history for pitchers with at least 150 save tries.
Kimbrel, who earned a four-out save in Atlanta's 5-2 victory Friday night, got Chris Owings to fly out to start the ninth. But Inciarte walked and stole second. Hill sliced one just inside the right field foul line to bring Inciarte home.
Heyward's two-out shot to left field off Addison Reed was his second home run in as many nights and extended his hitting streak to 11 games.
Montero was 0 for 4 with runners in scoring position before he hit Varvaro's 2-1 pitch just over the wall in left.
Inciarte was called safe on a bunt to start the 11th, but Braves manager Fredi Gonzalez challenged and the call was reversed. It didn't matter as Arizona went on to score the winner.
Starters Ervin Santana of Atlanta and Wade Miley of Arizona had strong outings. Santana gave up only an unearned run in seven innings, scattering six hits. Miley allowed two runs, one earned, on six hits in seven innings, fanning six with no walks.
The Braves took a 2-1 lead when, with two outs in the sixth, Justin Upton singled to right, then scored from first on Evan Gattis' second double of the night.
Atlanta second baseman Dan Uggla committed two errors, one leading to a run.
Martin Prado led off the second with a single, took second on David Peralta's base hit, then scored when Uggla threw the ball away trying to convert a double play on a fielder's choice ground out by Inciarte.
Atlanta tied it in the third when Andrelton Simmons doubled, took third on the right fielder Parra's errant throw, then scored on Heyward's grounder, sliding in barely below the catcher Montero's tag after a high throw from the third baseman Prado.
NOTES: Peralta has a hit in each of his first six big league games. ... In Sunday's series finale, Arizona's Chase Anderson (4-0, 3.32 ERA), winner of his first four major league starts, tries to make it five, while the Braves counter with Aaron Harang (4-4, 3.24). ... It was Prado's bobblehead night.