Kick It Out chairman Lord Ouseley believes that the careers of World Cup referees will be over if they fail to act should incidents of racism occur at the tournament.
Last week, FIFA president Gianni Infantino gave officials the power to abandon matches if players experience discrimination in Russia.
Infantino says there are measures in place should anything occur during matches and has warned of "serious consequences" if there are such issues.
Ouseley told Sky Sports News: "If players feel that something isn't happening and the referees should be taking action - the referees are aware their career is over.
"There are strong sanctions being taken. The referees have a responsibility on the pitch for the care and well-being of all the players whether it's because of an injury sustained due to normal action or something thrown onto the pitch or it's abuse."
When discussing Danny Rose's statement that he would not want his family in Russia, Ouseley added: "My view is that it's something you've got to take seriously.
"If you are a person with family and responsibilities you've got to think of your future and well-being.
"It's very disappointing that people cannot attend such a global event where everyone should be having a good time because of the fear of your own safety and security."