Arsenal opened the scoring after 58 seconds, Gabriel Martinelli sweeping home a perfect pass from captain Martin Odegaard, but Liverpool grew into the game and Darwin Nunez levelled in the 34th minute.
There was a lengthy break midway through the first half as Liverpool's Trent Alexander-Arnold received treatment for an ankle injury sustained in a tangle with Martinelli, and the England defender had to be replaced by Joe Gomez at halftime.
By then Arsenal were back in front, sweeping forward on the counter-attack as Martinelli turned provider, whipping the ball across the box for Saka to poke home a simple finish in first-half stoppage time.
Odegaard missed a golden opportunity to increase Arsenal's lead after the break and two minutes later Liverpool's Diogo Jota found Firmino with a perfectly weighted through ball for the Brazilian to slot across Aaron Ramsdale and into the net.
Arsenal came blazing back and a wild series of shots and blocks ended in a penalty when Liverpool's Thiago Alcantara was adjudged to have fouled Gabriel Jesus and Saka made no mistake from the spot to put his side 3-2 up in the 76th minute.
It was enough to seal a win that takes Arsenal back into first place on 24 points, one ahead of second-placed Manchester City after nine games.
Liverpool's disappointing start to the season continued, and their second league defeat leaves them in 10th spot on 10 points from eight games.
Arsenal coach Mikel Arteta was delighted with how his side responded to Liverpool's repeated comebacks.
"The threat that they have in any moment, they can open you up and they can exploit the spaces. I think they were very efficient the chances that they had," he said
"But I think we showed a lot of composure and character in difficult moments to overcome them, especially the way we played in the second half," he added.
Liverpool, in 10th place with 10 points after eight games, are enduring their worst start of Klopp's seven-year reign and with second-placed Manchester City set to visit Anfield next Sunday, there is little time to put things right.
"We don't have to look yet at the table but we know where we are without looking at it, so we are not done," Klopp said after his side's second defeat of the season.
Those two losses and four draws have left Liverpool trailing in the wake of Arsenal and City and though Klopp is frustrated by his side's slow start, he is not looking for excuses.
"This game (against Arsenal) is gone now, we cannot just accept a situation and say it's just not happening or whatever, we're not kids," he explained.
"We did a lot of good things, scored two really nice goals and played really good stuff against the team in form in the league, because we are probably the team not in form in the league."
Klopp's side face a quick turnaround as they visit Rangers in the Champions League in midweek before welcoming City and their free-scoring striker Erling Haaland to Merseyside next weekend.
Asked what he would be doing to lift his side ahead of those two matches Klopp said he would be getting back to basics.
"Working, talking, thinking probably first, and then talking to the boys, and then coaching them and helping them, and then we go again," he said.