Ulster captain, Alan O'Connor, scored a try in the 78th minute of the match, with Nathan Doak converting to edge the home side to a much-needed victory.
Scarlets had edged themselves in front with a 23-28 lead in the closing stages, but Ulster defended their late attacking spell well and went up the other end to grab a vital win.
In the Origin Round of the URC, the Ulster squad wore their junior club or school socks as a way to mark their own rugby origins.
The contest got off to the worst possible start for the home side, as the Welsh side scored two quickfire converted tries in the opening 11 minutes through wing Macs Page and full-back Ioan Nicholas.
Ulster battled their way back into the match after a clever lineout routine, involving James McNabney jumping and offloading to Nick Timoney, who was able to power over the driving maul. Murphy's conversion brought the score to 7-12.
Ulster were building more attacks as the province gained momentum.
Jack Murphy slotted home his first penalty of the evening to narrow the scoreline to 10-12.
Murphy then went for the spectacular and fired in a superb long-range penalty to put Ulster into the lead for the first time.
That lead became short lived, after Ulster failed to find touch and gave away a penalty with the clock in the red.
Ioan Lloyd's penalty gave the welshmen a slender lead of 13-15 at HT.
HT: Ulster 13-15 Scarlets
The second half started off with the two fly-halves adding another penalty each, as both sides looked to put on scoreboard pressure.
With the score at 16-18, Ulster went on the attack in the opposition 22.
After a series of penalties close to the Scarlets tryline, prop Callum Reid was able to muscle over from close range for Ulster's second try.
Murphy's conversion took the game to 23-18 for the home side.
Ulster were then left kicking themselves after a misplaced pass gave Vaea Fifita the opportunity to break Ulster's defensive line and run through for an easy score. With the score now at 23-25, Ulster were back to square one and needed to regroup.
Ulster were put under serious pressure from the away side who wanted to close out a victory. Ulster's defence remained solid and they were able to hold off the Welsh side just millimetres away from their own tryline.
Back-rower, James McNabney showed a rare boot, as he was able to hack the ball away up the field.
The Welsh side then added another three points from the tee to make the scoreline 23-28 and with Ulster running out of minutes.
Then came the late show. Ulster drove into the Scarlets 22 and went close through a powerful maul from a lineout.
Ulster kept the ball alive and pummelled the Scarlets tryline, with captain Alan O'Connor, getting the crucial try in the 78th minute.
Nathan Doak's conversion made the difference in the scoreline and Richie Murphy's side celebrated a gritty victory against their Welsh opposition at Kingspan Stadium.
Player of the Match - James McNabney
A superb display from the number eight, with his determination, ball-carrying and defensive efforts making a huge impact in Ulster's victory.
Scorers
Ulster:
TRY: Timoney, Reid, O'Connor
PEN: Murphy X3
CON: Murphy x2, Doak
Scarlets:
TRY: Page, Nicholas, Fifita
PEN: Lloyd x3
CON: Lloyd x2
Teams
Ulster: 15 Stewart Moore, 14 Michael Lowry, 13 James Hume (Postlethwaite), 12 Stuart McCloskey, 11 Jacob Stockdale, 10 Jack Murphy (Flannery), 9 Conor McKee (Doak), 8 James McNabney, 7 Nick Timoney, 6 Matty Rea (McCann), 5 Matthew Dalton (Treadwell), 4 Alan O’Connor (c), 3 Scott Wilson (O'Toole), 2 Rob Herring (Andrew), 1 Callum Reid (Crean).
Replacements: 16 John Andrew, 17 Sam Crean, 18 Tom O’Toole, 19 Kieran Treadwell, 20 David McCann, 21. Nathan Doak, 22 Jake Flannery, 23 Jude Postlethwaite.
Scarlets: 15 Ioan Nicholas, 14 Macs Page, 13 Joe Roberts, 12 Johnny Williams, 11 Tomi Lewis, 10 Ioan Lloyd, 9 Gareth Davies, 8 Jarrod Taylor, 7 Dan Davis, 6 Max Douglas, 5 Sam Lousi, 4 Alex Craig, 3 Sam Wainwright, 2 Ryan Elias (c), 1 Alec Hepburn.
Replacements: 16 Marnus van der Merwe, 17 Sam O’Connor, 18 Gabe Hawley, 19 Jac Price, 20 Vaea Fifita, 21 Archie Hughes, 22 Charlie Titcombe, 23 Steff Evans.
Referee: Sam Grove-White (SRU)
Assistant Referees: Padraic Reidy (IRFU), Paul Haycock (IRFU) .
TMO: Rob McGreer (SRU)