Serbia had dominated against a Costa Rica side who rarely threatened after a bright start, but chances were thin on the ground until Kolarov made the breakthrough in the 56th minute.
Tadic, 29, came into the tournament after an inconsistent season, but one he finished strongly, providing three assists in his final seven games. Costa Rica had a few chances at the goal but failed to convert them. Serbia now leads Group E with three points, ahead of pre-tournament favorite Brazil.
With Brazil heavy favorites to win the group, which also features Switzerland, the victory delivered a potentially vital three points for Serbia and lifted a weight from coach Mladen Krstajic, in what was his first competitive game.
The Ticos were the surprise of the last World Cup, sweeping through to the quarterfinals in Brazil before being ousted by the Netherlands on penalties.
"There's always a lot of talk about four years ago, but this is a new World Cup, a new history", Costa Rica defender Giancarlo Gonzales said.
"We dedicate this win to our Director, and we want him to know that we are all supporting him", said Kolarov, referring to the 45-year-old Bunjevcevic, who assumed the post with the Serbian federation on August 2016. It's a World Cup, here there are the best national teams around the world and we are one of them.
Note that our website will be available a text online broadcast of the match Costa Rica - Serbia.
It is very hard to see the Germany v Mexico match going any other way that a victory for Germany, and I expect from the very minute the match kicks off it will be Germany that will be taking the lead.
We have been treated quite well in the early stages of this World Cup.
Milinkovic-Savic spent the bulk of the first half shouting because he didn't get the ball where he wanted it, and too many crosses missed the mark because the intended target took a different route.