Israelis and Palestinians are trying to agree a long-term deal on Gaza’s future and end weeks of deadly fighting before a five-day ceasefire expires.
On Sunday, Israel’s prime minister said any agreement had to meet its security needs and warned Hamas that it faced “harsh strikes” if rocket fire resumed.
A leader of the Islamist group insisted the Palestinian delegation would “not cede any of the rights of our people”.
Officials say that 2,016 Palestinians and 66 Israelis have died since 8 July.
The current ceasefire came into effect on Wednesday, building on a previous three-day halt in the hostilities. But it is due to expire at midnight (21:00 GMT) on Monday.
Negotiating teams returned to Egypt’s capital Cairo for indirect talks on Sunday following consultations.
The Egyptian mediators hope both sides will agree to the ceasefire being extended, allowing them to resolve outstanding issues at a later date.
But late on Sunday, an unnamed Palestinian official told the Egyptian state news agency, Mena, that Israel’s position had retreated “from what had already been achieved and discussions had returned to square one”.
At his weekly cabinet meeting, Israeli Prime Minister Benjamin Netanyahu said: “Only if there is a clear response to our security needs will we agree to reach understandings.”
“If Hamas thinks that through continued intermittent firing it will cause us to make concessions, it is mistaken. For as long as quiet does not return, Hamas will continue to absorb very harsh strikes.”
Palestinian negotiator Qais Abdul Karim said Israel was seeking guarantees that Hamas and other factions in Gaza would be disarmed, while the Palestinians were demanding an end to the Israeli and Egyptian blockades of Gaza without preconditions.
Hamas has stated that it will not give up its weapons, while Israel has said it needs to maintain some control over Gaza’s crossings to prevent the smuggling of weapons.
Hamas’ deputy political leader, Moussa Abu Marzouk, warned: “The Palestinian delegation will not cede any of the rights of our people.”
Despite the apparent deadlock, Islamic Jihad representative Ziad Nakhleh said he expected the ceasefire to be extended.
“The war is behind us now,” he told AP. “We are not returning to war.”