It was an unfortunate 17 - 16 defeat for the Ospreys on Friday.
With the Ospreys seemingly having battled back to steal the game at the death, thanks to Shaun Connor's third penalty, the visitors were penalised at the restart when prop Paul James was sin-binned for pole-axing Matt McCullough.
The resulting penalty left Ulster's rookie fly-half, Niall O'Connor, to calmly slot the points.
But, deep in injury time, the Ospreys had another chance to snatch the result with a late penalty, but Connor went wide keeping the final score 17 -16.
Ulster took an early lead through O'Connor, and could have had a try when prop Tom Court was put through from turnover ball, were it not for some strong Ospreys defending taking him into touch.
However, Ulster built on their missed opportunity with O'Connor's second penalty, just after the first quarter.
The Ospreys fly-half did manage to successfully convert their next chance, Connor calmly kicking the penalty, but Ulster finished the half by furthering their lead with O'Connor's third penalty.
Ulster furthered their lead shortly after the restart when Scotland international Simon Danielli crossed in the left corner for an unconverted score but, just before the hour, Connor charged down his opposite number and put Andrew Bishop clear for a try which Connor easily converted.
The Ospreys fly-half then reduced Ulster's lead to a single point with another penalty before, four minutes from the end of normal time, his third penalty gave the Ospreys a lead before the tense finish.
The Scorers:
Ulster:
Try: Danielli
Pens: O'Connor 4
For Ospreys:
Try: Bishop
Con: Conor
Pens: Conor 3
The teams:
Ulster: 15 Mark Bartholomeusz, 14 Simon Danielli, 13 Tommy Bowe, 12 Seamus Mallon, 11 Mark McCrea, 10 David Humphreys, 9 Kieran Campbell, 8 Roger Wilson (c), 7 Kieron Dawson, 6 Matt McCullough, 5 Ryan Caldwell, 4 Tim Barker, 3 Tom Court, 2 Nigel Brady, 1 Justin Fitzpatrick.
Replacements: 16 Matt Miles, 17 Declan Fitzpatrick, 18 Justin Harrison, 19 David Pollock, 20 Paul Marshall, 21 Niall O'Connor, 22 Bryn Cunningham.
Ospreys: 15 Lee Byrne, 14 Stefan Terblanche, 13 Mark Taylor, 12 Andrew Bishop, 11 Aled Brew, 10 Shaun Connor, 9 Justin Marshall, 8 Filo Tiatia (c), 7 Steve Tandy, 6 Tom Smith, 5 Mike Powell, 4 Brent Cockbain, 3 Cai Griffiths, 2 Barry Williams , 1 Paul James.
Replacements: 16 Richard Hibbard, 17 Andrew Millward, 18 Andy Lloyd, 19 Ben Lewis, 20 Martin Roberts, 21 Gavin Henson, 22 Jonathan Vaughton.
Venue: Ravenhill
Referee: David Changleng
Touch judges: E O'Dowd, B Macneice