The ruling Bharatiya Janata Party has won only about 39 seats and the regional Janata Dal (Secular) won 40 seats.
Karnataka, home to India's information technology industry, has been the only southern state to be ruled by the BJP.
The BBC's Soutik Biswas in Delhi says the results in Karnataka came as a big blow to the Hindu nationalist BJP.
Our correspondent says that they are a setback to its hopes to win the general elections next year.
The party has lost ground there amid allegations of corruption against senior state party leaders.
At the same time the results are a boost to the Congress party, our correspondent says, which is fighting a number of corruption scandals nationally while also struggling with a slowing economy.
Last November, the BJP suffered a major setback after senior leader and former state Chief Minister BS Yeddyurappa formed his own party.
Mr Yeddyurappa quit as chief minister after an anti-corruption report in 2011 accused him of being involved in a mining scandal.
An anti-corruption report in that year alleged that the scam cost the exchequer more than $3bn (£1.8bn) between 2006 and 2010. Mr Yeddyurappa denied any wrongdoing.
"We have had a setback," former Karnataka chief minister and BJP leader Sadananda Gowda said.
The BJP won 110 seats in the last state elections in 2008.
Analysts say it took the BJP a lot of time and effort to build the party in Karnataka, and those achievements have been squandered away in one term.
Senior Congress party leader Siddaramaiah said his party was "confident of forming the next government on our own".
More than 71% out of 41 million eligible voters cast their votes on Sunday and more than 2,900 candidates contested the polls.
Ballots are being counted at 36 centres, with complete results expected later.

bbc.com