Speakers at the rally largely focused on Netanyahu's ongoing legal troubles as grounds for him to step down, according to Israeli newspaper Haaretz.
Netanyahu was slapped with corruption charges by Attorney General Avichai Mandelblit after a lengthy investigation into the PM's alleged influence-peddling and accepting of expensive gifts.
The rally took place days after Tel Aviv saw a smaller rally in support of Netanyahu. The Tuesday demonstration saw about 2,000-3,000 people, mostly elderly supporters of Netanyahu's Likud party.
Netanyahu's position appears weaker than it has been in years as the Knesset, the Israeli parliament, may be dissolved if Netanyahu's right-wing Likud party and the opposition Blue and White alliance does not manage to form a unity government by the deadline of 11 December.
In April, Netanyahu's party won the poll but fell short of obtaining the majority needed to form a government. In the second election held in September, the Likud Party also failed to secure a majority in the Knesset.