The inquiry into the former head of the Catholic Church in Scotland is not likely to begin until after a new Pope is chosen – a process starting later.
On Sunday, the cardinal apologised to the Church and the people of Scotland.
He resigned after three priests and a former priest alleged improper behaviour dating back to the 1980s.
Cardinal O’Brien was Britain’s most senior Roman Catholic cleric until he stood down as the Archbishop of St Andrews and Edinburgh last month.
BBC religious affairs correspondent Robert Pigott says the inquiry into his conduct will be an internal Church investigation and the results might not be made public.
It will be carried out by the Vatican under a new Pope, not by the Church in Scotland, and any punishment will depend on the circumstances of his improper sexual conduct.
BBC correspondent adds that the statement issued by Cardinal O’Brien on Sunday lacked specific detail, but its acknowledgement of sexual misconduct exposes him to charges of hypocrisy from critics with respect to his outspoken attacks on homosexual behaviour in the past.
Cardinal O’Brien’s resignation took place after the former priest and three current priests from the diocese of St Andrews and Edinburgh complained to the Pope’s representative to Britain, Archbishop Antonio Mennini, in early February about what they alleged had been inappropriate behaviour towards them in the 1980s.
In his statement, issued through the Roman Catholic Church in Scotland, Cardinal O’Brien said: “In recent days, certain allegations which have been made against me have become public. Initially, their anonymous and non-specific nature led me to contest them.
“However, I wish to take this opportunity to admit that there have been times that my sexual conduct has fallen below the standards expected of me as a priest, archbishop and cardinal.
“To those I have offended, I apologise and ask forgiveness. To the Catholic Church and people of Scotland, I also apologise.
“I will now spend the rest of my life in retirement. I will play no further part in the public life of the Catholic Church in Scotland.”
Sources in the Scottish Catholic Church told the BBC that Cardinal O’Brien had at first thought he might eventually be able to return to a role in Church life after retirement.
The cardinal, who had initially said he was taking legal advice when the allegations against him were made public, had been due to retire later this month when he turned 75.
Cardinal O’Brien, who was born in Ballycastle, Co Antrim, had been the Archbishop of St Andrews and Edinburgh since 1985.
He has been an outspoken opponent of plans to legalise same-sex marriage in Scotland and was named “bigot of the year” by gay rights charity Stonewall last year.
In an interview with BBC Scotland shortly before the allegations against him were made public, the cardinal said he believed priests should be able to marry and have children if they wished.
He said it was clear many priests struggled to cope with celibacy. – BBC