Prime Minister Tony Abbott has admitted he should not have winked at a radio host when a sex hotline worker called to ask him about the budget.

Mr Abbott also brushed aside leadership speculation after disastrous recent polls, saying he expected to remain Coalition leader into the next election.

The wink, which dominated debate on social media on Wednesday, came on a torrid day for the Prime Minister, who was forced to defend his daughter Frances for receiving a $60,000 design degree scholarship from an institution linked to a Liberal donor.

Mr Abbott said on Thursday morning that he was responding to the ABC radio host when he winked during the interview.

"Look, I was looking at Jon Faine, he was smiling at me and I winked back at him," Mr Abbott told Channel Nine.

"I shouldn't have done it, Karl. I should've been more focused on the caller and more focused on the interview."

Mr Abbott said he was "momentarily distracted" by Faine.

Asked if he was being disparaging towards the woman, Mr Abbott said he was attempting to engage with the caller.

''Mistakes are always regrettable . . . and I will do my best having made a mistake yesterday to make none today,'' he said.

On Thursday during his radio program, Mr Faine rejected comments that he winked or smirked at Mr Abbott.

''I raised an eyebrow because it's not every day that a grandmother on radio says that she supplements her pension by working on a sex call line,'' he said.

''It is a complete fabrication to say I somehow winked at the prime minister or was smirking or anything like that. Nor was I indicating that I needed his permission to continue which was the statement from the Prime Minister's office.''

Earlier he told ABC TV: "Whether it was a school boy snigger at the mention of sex or a Benny Hill moment - everyone who watches it will make up their own mind. But I do think it's got way out of all proportion."

The issue erupted after, Gloria from Warburton, called into Faine's program to speak to the Prime Minister and said she had to work on a sex hotline to make ends meet. The ABC did not know she worked on a sex line before the call went to air.

"I'm a 67-year-old pensioner. Three chronic incurable medical conditions, two life threatening. I just survive on around $400 a fortnight after I pay my rent," Gloria said. "I work on an adult sex line to make ends meet. That's the only way I can do it."

Faine told ABC TV that Mr Abbott seemed more nervous than usual during the interview – in which Mr Abbott contradicted his government's higher education policy.

"He did seem very nervous and I don't know why," he said. "He's the Prime Minister, he's been in public life for ages.

While some social media users were angry at the wink, the caller herself seemed to take the furore in good humour.

In a follow-up call to the ABC on Thursday, Gloria described the incident as ''sleazy [and[ slimy''.

But she said her new-found fame had not helped her telephone sex career.

''It didn't improve my call rate - I only got one call and it only lasted for four minutes,'' she said.

Foreign Minister Julie Bishop, the only woman in Mr Abbott’s cabinet, said the wink was not sexist.

''For someone to wink to say 'It's ok mate' - I don't find anything sexist about that and I think that people are far too quick to try and judge others through this prism of everything being about misogyny or sexism,'' she said.

''I don't read it that way . . . Believe me, I've worked in some sexist environments and this [government] is not one of them.''

Opposition Leader Bill Shorten said on Thursday that the wink showed how out of touch the Prime Minister was

''I'm sure the Prime Minister's minders would've had a collective forehead-slapping moment, and said, 'oh my goodness Tony's at it again','' Mr Shorten told reporters in Melbourne.

''I don't know what was going through his mind then. It just shows how out of touch the prime minister is.''

The Greens have called on Mr Abbott to step down as Minister for Women and appoint a woman to the role after the winking controversy, but Mr Abbott said he on Thursday that he would not resign from the position.

In a statement on Thursday, Senator Larissa Waters, the party's spokesperson for women said: ''The Prime Minister’s latest gender faux pas, dismissing an elderly woman's work on a sex line to cope with financial burden, by winking inappropriately, is further evidence that he is not a fit Minister for Women.

''We need a Minister for Women who is actually a woman and who is prepared to stand up for women – not simply dismiss their problems with a wink and a smirk.''

Mr Abbott was also on the defensive over reports, first revealed in The Guardian, that Les Taylor, the chairman of the Whitehouse Institute of Design board of governors, had recommended Mr Abbott's daughter Frances for a $60,000 design degree scholarship. Mr Taylor is a Liberal party donor.

Mr Abbott said on Thursday that his daughter was awarded the scholarship on merit and maintained it on merit.

"I'm very proud of her; she's doing a fantastic job," he said.

"I've always said families should be kept out of the front line. That's the way I've always tried to run my political operation – that we play hard but fair. Families should be [left] out of it."

The Prime Minister also said voters did not expect an ''action replay of the last six years'' over leadership, saying he ''expected to remain as leader'' to the next election in 2016.

with AAP

Follow us on Twitter