All perfect praise be to Allah, The Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad, sallallahu ‘alayhi wa sallam, is His slave and Messenger.
Since your father gave you this money for that particular purpose and you spent it for another purpose, you should tell your father about it. If he approves of your action, then there is no problem with that and you would not be obliged to give him back the money. If he does not approve of it, then you must give it back to him.
If your mother has given you this pocket money without specifying how you should spend it, then there is no harm on you in giving back your father's money from it.
Allah knows best.