Richie Murphy’s side rallied after a 10 point deficit at half time to leave Newport with maximum points.
The hosts broke the dead lock with less than a minute on the clock.
The Dragons reclaimed the ball from the kick off, they probed down Ulster’s right flank before switching the ball across the field and Will Reed released centre, Aneurin Owen, and the Dragons captain stretched over the whitewash to dot down.
Reed added the conversion.
Ulster levelled the game on 10 minutes, Jacob Stockdale won a jackal penalty andJack Murphy kicked for the corner. From the lineout, the forwards set up a maul and Ireland internatonal, Rob Herring, peeled off and surged for the line but was stopped short. The ball was quickly recycled and Nathan Doak hit James Hume and the centre finished.
Doak added the conversion, but Ulster were reduced to 14 men four minutes later as Herring was yellow carded for a head-on-head contract with Matthew Screech.
Reed nudged the home side back in front from the resulting penalty.
Full back Huw Anderson extended the home side’s lead when he finished from close range and Reed converted.
Dragons' Joe Westwood was yellow carded for slowing the ball on 28 minutes and Ulster immediately made their numerical advantage count. The penalty was kicked to the corner and from the lineout maul, Herring stormed over with Doak converting.
Reed was off target with a 32nd minute penalty.
Despite having a man in the sin bin the Dragons got their third try on 34 minutes, Owen made a break down the wing, the ball was worked inside, Anderson glided through the defence and popped the ball to scrum half Rhodri Williams for an unopposed run in.
Redd’s conversion gave the Dragons a 24-14 half time lead.
Ulster came out of the changing rooms fired up and it took them just three minutes after the restart to get their third try.
From a tap penalty the forwards probed at the line, Tom O’Toole was stopped just short, but Herring squeezed over for his second try of the evening with Doak adding the conversion.
Reed was on target with his second penalty four minutes later. but the out half was unsuccessful with another attempt of the tee on 52 minutes.
Ulster were unlucky not to get a fourth try on the hour mark, the forwards battered at the line from a tap penalty and as Scott Wilson powered for the line the referee adjudged the replacement prop held on in the tackle and awarded the home side a penalty.
Reed put the Dragons nine points ahead with 17 minutes remaining with his third penalty.
Replacement scrum-half, John Cooney, then landed a penalty from the Dragons 10 metre line two minutes later to bring Ulster back within a converted try.
Minutes later, the Dragons won another penalty around a ruck and Reed's strike struck the upright with a 68th minute penalty.
Ulster took the lead on 71 minutes and sealed a bonus point with a fourth try thanks to a superb moment from Scott Wilson.
Anderson tried to run the ball out of his own 22 and Wilson managed to make a turnover and the prop instantly sprinted over the tryline, leaving Cooney with a simple conversion.
Hooker John Andrew was denied a try in the last minute for an obstruction in the build- up but Ulster were playing with a penalty advantage and Cooney then ran down the seconds and added an extra three points with the last kick of the game.
FT: Dragons 30-34 Ulster
Player of the Match - Scott Wilson
A big impact off the bench with his carrying. Made a big turnover and try that grabbed Ulster momentum and a bonus-point.
Scorers
Dragons
TRY: Aneurin Owen, Huw Anderson, Rhodri Williams
CON: Will Reed x 3
PEN: Will Reed x 3
Ulster
TRY: James Hume, Rob Herring x2, Scott Wilson,
CON: Nathan Doak x 3, John Cooney
PEN: John Cooney x2
Teams
Dragons:
H Anderson; R Dyer, J Westwood, A Owen (capt), A Hewitt; W Reed, R Williams, A Wainwright, H Keddie, S Lewis-Hughes, G Nott, M Screech, C Coleman, E Dee, R Martinez
Replacements:
B Coghlan for Dee 62 mins, D Kelleher-Griffiths, P Latu for Coleman 53 mins, S Cummins for Screech 14 mins, T Basham for Keddie 62 mins, D Blacker, J Thomas, J Rosser for Westwood 69 mins
Ulster:
S Moore; M Lowry, J Hume, S McCloskey, J Stockdale; J Murphy, N Doak, J McNabney, N Timoney, D McCann, M Dalton, A O’Connor (capt), T O’Toole, R Herring, C Reid
Replacements:
J Andrew for Herring 66 mins, S Crean for Reed 48 mins, S Wilson for O’Toole 52 mins, K Treadwell for Dalton 64 mins, R Crothers for McNabney 69 mins, J Cooney, for Doak 56 mins A Morgan, Z Ward for Stockdale 69 mins