Billed as a business breakfast, Mr Brokenshire was listed as guest speaker alongside DUP leader and First Minister Arlene Foster.
However, Mr Brokenshire withdrew in the face of unwelcome publicity.
SDLP leader Colum Eastwood said he needs to show he is not favouring one Northern Ireland party over any other.
The story was first reported in the Irish News on Wednesday.
"It's quite clear, given the Tory party conference, champagne receptions and James Brokenshire agreeing to attend a fundraiser, that they seem to be working very closely together," Mr Eastwood told the BBC.
"That's fine on one level, but the British secretary of state and the British Government should not be in the pocket of any one party.
"He needs to now go overboard in demonstrating that he isn't captured by the DUP, that he is his own man and he's prepared to stand up for what's right in Northern Ireland - not for what Arlene Foster tells him."
People attending the event at a café in Lisburn, County Antrim, are being charging £30 a plate or £300 for a table of 10.
The invitation described it as "an excellent opportunity to network with business and elected representatives".
An NIO spokesman said that it does not "discuss [Mr Brokenshire's] diary and he is not attending the event."
The DUP refused to comment.
Last week, the DUP held a champagne reception at the Conservative conference amid claims the two parties have done a deal in the wake of the Brexit vote.
On Wednesday, the secretary of state met DUP MPs in what was described as a constructive and useful engagement.
The NIO said it was part of a series of meetings, with the MPs of all the Northern Ireland  parties represented at Westminster.
The Northern Ireland Secretary has pulled out of an event run by the DUP after it was revealed to be a party fundraiser.