What's this thing called?

An eraser.

"Wipe off" ("Rub off" means something completely different)

"Felt erasers are used for marks on a chalkboard or whiteboard."

http://en.wikipedia.org/wiki/Eraser#Other

"The chalk marks can be easily wiped off with [...] a special blackboard eraser."

http://en.wikipedia.org/wiki/Chalkboard#Design

It's called an eraser, and the 1st phrase is correct.

Eraser. Ok so this is weird in that if you use those plastic eraser things, then you say please erase the board. THEN, if you use a tissue, cloth, or anything of that sorts, you can ALSO say please wipe off the board.
Rub off sounds weird and unnatural. Usually that implies something that is harder to get off than usual.

I'm pretty sure its called a chalkboard/whiteboard eraser.

We usually don't "rub" a whiteboard. Usually people would say "please erase the board." :)

Are you writing a story? If so, Good luck!

Usually just called an "eraser". The name name given to the piece of rubber on top of a pencil, even though they're very different objects. It is a thing which erases, so an "eraser".

If you want to be more specific, you can say "a pencil eraser", "a whiteboard eraser", etc.

And it would be "wipe off". The object of "rub off" is the substance you are removing. "Can you rub that sticky stuff off the whiteboard?"

the thing you use is an eraser.
Please wipe off sounds better