MPs have rejected Mike Ashley's invitation to visit the retailer's main Shirebrook warehouse prior to his appearance before the committee on 7 June.
The Sports Direct Boss has threatened to pull out and is looking for legal advice about his scheduled meeting with the parliamentary select committee.
Chairman of the House of Commons Business select committee Iain Wright informed Ashley in a letter that he expects him to attend his committee as agreed, but his members would not visit the Sports Direct site.
The letter stated: "the committee has considered carefully your repeated and conditional request that we visit Shirebrook before this date” however the committee " do not accept that witnesses attach conditions to attending in response to a summons”.
Wright added that while he expects to see Ashley at the committee next month, he is permitted to bring a legal representative and he may visit the Sports Direct warehouse after hearing his evidence.
The MP warned that if Ashley fails to attend, more severe proceedings are likely to follow. "I am sure that neither of us wants to engage in a potentially lengthy and public process that would follow any non-attendance.”
A Sports Direct Spokesman noted that Ashley is "disappointed that members of the committee have chosen not to visit Shirebrook in order to see working conditions with their own eyes, as we continue to believe that this is obviously the best way forward".
"The committee are well aware that Mike Ashley's proposal to attend Westminster on June 7 was made on the understanding that MPs would visit Shirebrook on any day in advance of June 6."
"The committee are clearly being unfair, Mike will therefore now seek legal advice with regard to his position in relation to June 7," the spokesman added.