All perfect praise be to Allaah, The Lord of the worlds. I testify that there is none worthy of worship except Allaah and that Muhammad is His slave and Messenger.
You are not Islamically obliged to pay the debts of your husband, and the authorities have no right to force you. So you are permitted to prevent this by doing what you mentioned.
The validity of the marriage does not require registering it with the official authorities although we always recommended that it be registered in order to preserve one's rights and ward off harm.
It should be mentioned that if the government dropped a portion of the debt from that man on condition that he did not have a working wife, then he is not permitted to conceal the truth about this, as the Prophet said:,"Muslims are bound by their conditions." [Abu Daawood - As-Suyooti graded it saheeh (sound)]
Allaah knows best.