Greek centre-right leader Antonis Samaras has said he cannot form a coalition government, dealing a blow to supporters of the foreign bailouts.
His New Democracy emerged as the biggest party after Sunday's vote, but he said a coalition was "impossible".
Far-left group Syriza, which is opposed to austerity measures, will now try to form an anti-bailout coalition.
The Greek result combined with France electing an anti-austerity president has caused alarm among EU leaders.
German Chancellor Angela Merkel said Greece's austerity reforms were of "utmost importance".
European Commission spokeswoman Pia Ahrenkilde Hansen said Brussels hoped and expected that the future government of Greece would respect Greece's prior engagement.
The markets had slumped following the election results, but largely recovered later. The Athens stock exchange, however, plunged 6.67% by the end of the day.'Barbaric' measures
In return for two EU/IMF bailouts worth a total of 240bn euros (£190bn; $310), Greece agreed to make deep cuts to pensions and pay, raise taxes and slash thousands of public sector jobs.
The financial chaos sparked huge social unrest, and led to a deep mistrust of the parties considered to be the architects of austerity.
Mr Samaras said his party had done "everything possible" to form a government.
"I tried to find a solution for a government of national salvation, with two aims: for the country to remain in the euro and to change the policy of the bailout by renegotiation," he said in a televised address.
"We directed our proposal to all the parties that could have participated in such an effort, but they either directly rejected their participation, or they set as a condition the participation of others who did not accept."
President Karolos Papoulias has now arranged a meeting for Tuesday morning with Alexis Tsipras, whose Syriza coalition came second in Sunday's election.
Mr Tsipras will be given three days to negotiate a coalition, and he promised to stitch together a left-wing coalition to reject the "barbaric" measures associated with the EU/IMF bailout deal.
"We will exhaust all possibilities to reach an understanding, primarily with the forces of the left," he said.
But analysts say he is likely to struggle to reach the numbers needed for a cross-party majority.
The BBC's Mark Lowen in Athens says Greece's political crisis is deepening, and the likelihood of fresh elections is growing ever stronger.
Despite emerging as the biggest party, New Democracy's support slipped from 33.5% in the last election to less than 19% on Sunday.
Support for the centre-left Pasok, which also supported the austerity measures, plummeted from 43% to just over 13%.
The major winners were anti-bailout groups - Syriza, with 16.8%, and the ultra-nationalist Golden Dawn with almost 7%.