Whyte and Chisora will go head to head on Saturday night two years after Whyte defeated Chisora the first time round.
Joshua, believes it will be Chisora who will beat Whyte in the second fight after going the distance when they first met.
Whyte is a potential rival for Joshua in 2019, however 'The Body Snatcher' believes after he is victorious against Chisora a second battle with AJ will happen soon.
"I can't bank on what he's doing, I've got to focus on my own career," Whyte told Sky Sports.
"It would be great if we could get the rematch on, but it's only a matter of time.
"There's only six or seven top guys around and I have consistently proved I am of that level and I am the one getting better and better. It's only a matter of time."
Whyte isn't too worried that the Joshua is supporting Chisora, he said "They're both Finchley boys and they're mates, but I bet he said it with a stupid smile on his face,"
"It's cool though. Listen, I've been fighting all my life, climbing this hill my whole life, and I'm not looking for sympathy
"Nothing great comes easy. If it comes easy everybody would be doing it."