The province had put in an impressive first half performance to go into the break 0-19 up, however the welshmen made the most of a fast start to the second half to snatch the win in front of a loud home crowd in the Welsh capital.
Ulster made a good start, with early attacking phases being held at bay by a resolute Welsh defence. A series of penalties saw the referee give a Cardiff player a yellow card as Ulster piled on the pressure. Their defence was breached when blindside flanker Cormac Izuchukwu muscled his way over the line for the first score.
Nathan Doak converted the try.
Ulster's second would come a few minutes later, after a well worked move that began with Nathan Doak breaking in-field and Jude Postlethwaite was sent clear to finish. Doak again converted the try.
Postlethwaite then scored a brilliant team try that began with Werner Kok showing his athleticism to keep the ball in-play before Stuart McCloskey pulled off a clever chip that fell into the path of Postlethwaite, who dived over for his second and Ulster's third.
Ulster received a blow with captain Iain Henderson coming off earlier than expected, replaced by Harry Sheridan.
Cardiff started to have more possession of the ball and threatened before half-time but Ulster went into the break with their lead at 0-19.
HT: 0-19
The Welsh side started the half with speed and intent and immediately gave their home support a lift with entries into the Ulster 22.
But for a superb last-ditch tackle on Mason Grady into touch by Nick Timoney in Ulster's corner, the welshmen would have scored their first sooner.
The away side were guilty of giving away penalties and David McCann was shown a yellow card.
Cardiff took full advantage, working the ball wide to winger, Iwan Stephens, who dived over in the corner.
Sheedy added the extra two points.
Ulster went close to securing the all-important fourth bonus-point try through Harry Sheridan, however his dive for the score was in vain as the welshmen got back to hold it up.
It felt like a turning point and Cardiff swung the momentum back.
Cardiff started to win more penalties and kicked to the corner, backing their lineout mauls to do damage.
Replacement hooker, Daf Hughes, scored for the home side, after going over from the maul.
Minutes later, the home team would go in front for the first time, after Thomas Young dived for the line after another lineout maul in the Ulster 22. Sheedy's conversions increased the scoreline to 21-19.
With time running out, Nathan Doak took a long-range penalty from distance hoping to get Ulster back on top, however the kick was short of the posts and the home side would hold out for the comeback victory.
Ulster head back to Belfast with a losing bonus-point and disappointment after the lead held at half-time.
Teams
Ulster:
(15-9) Ethan McIlroy; Werner Kok, Jude Postlethwaite, Stuart McCloskey, Mike Lowry; Aidan Morgan, Nathan Doak;
(1-8) Eric O’Sullivan, James McCormick, Tom O’Toole, Iain Henderson (C), Kieran Treadwell, Cormac Izuchukwu, Nick Timoney, David McCann.
Replacements: John Andrew, Andrew Warwick, Scott Wilson, Harry Sheridan, Marcus Rea, John Cooney, Ben Carson, Ben Moxham.
Cardiff:
(15-9) Cam Winnett; Mason Grady, Rey Lee-Lo, Ben Thomas, Iwan Stephens, Callum Sheedy, Aled Davies;
(1-8) Ed Byrne, Evan Lloyd, Keiron Assiratti, Josh McNally, Teddy Williams, Ben Donnell, Dan Thomas, Alun Lawrence.
Replacement: Daf Hughes, Corey Domachowski, Rhys Litterick, Rory Thornon, James Botham, Thomas Young, Johan Mulder, Rory Jennings.