Retired Brigadier Gen. James Shelton has been friends with Petraeus for more than three decades and reached to out to him after he resigned from the CIA. Shelton told ABC News that the former four-star general wrote him a letter recently confessing to the affair.
Petraeus, 60, writes in the letter, "Team Petraeus will survive.... though [I] have obviously created enormous difficulty for us," according to Shelton.
A former spokesman for Petraeus told ABC News that fury was an inadequate description for Holly Petraeus after learning her husband of 38 years had an affair.
But in the letter, Petraeus writes that his wife is "...once again demonstrating how incredibly fortunate I was to marry her."
Shelton said he has shocked when news of the affair broke. Shelton says he has never met Broadwell but talked to her on the phone as she worked on the Petraeus biography, "All In." Broadwell thanked Shelton in the book's acknowledgments as "being wonderfully helpful."
Shelton says he found Broadwell engaging.
"I don't think she wove a web around Dave and dragged him in, I don't think that at all. I think it was mutual," Shelton told ABC News.
The disgraced general also stuck by his decision to step down as head of the CIA, writing, "I paid the price (appropriately) and I sought to do the right thing, at the end of the day."
Neither Broadwell nor Petraeus would comment when ABC News tried to reach them overnight.
However, there are many in Washington who now wonder if Shelton's talking about this letter is the beginning of a carefully choreographed campaign by Petraeus to rehabilitate his image.
Shelton says while he was disappointed in Petraeus' actions, he thinks it was a one-time mistake.
"I believe that Dave Petraeus was that kind of guy. He wasn't looking for it, it happened," he said.
While it is unclear who may have initiated the affair, what is clear is the scope of their relationship. An FBI investigation has uncovered hundreds if not thousands of emails exchanged between the two.
The 40-year-old author was stripped of her military security clearance after a federal probe alleged she was storing classified military material at her home.
The FBI found classified material on a computer voluntarily handed over by Broadwell earlier in the investigation.
Prosecutors will now have to determine how important the classified material is before making a final decision on how to proceed. Authorities could decide to seek disciplinary action against her rather than pursue charges.
Since announcing his resignation from the CIA last month, Petraeus has kept a low profile only appearing in closed door hearings before the House and Senate intelligence committees to testify about what he learned first-hand about the Sept. 11 attack in the U.S. consulate in Benghazi.
ABC News' Mosheh Gains contributed to this report.