I cant be charged with rape for this?

Rape is legally defined as unwanted sexual contact. If she didn't want to, and she tried to let you know that she didn't want to, then it's rape.