Praise be to Allah.
The Prophet (peace and blessings of Allaah be upon him) encouraged us to marry those who are religiously-committed, because basically women are weak and may change their beliefs and thoughts and even their religion for the slightest reason. So you should not have even considered marrying one whose religious commitment was not great, let alone marrying one who has no religious commitment at all, on the grounds that you would be able to guide him.
Marriage to an atheist is invalid and the marriage contract is basically null and void. It is not permissible for a woman who believes in Allaah and the Last Day to enter into such an invalid marriage on the grounds that this man may possibly be guided after marriage. She should have done what the great Sahaabi woman Umm Sulaym did when she refused to marry Abu Talhah – who was a kaafir at the time – unless he became Muslim, and he did so. This was the greatest mahr (dowry) in Islam as Anas (may Allaah be pleased with him) said. (al-Nasaa’i, 3341; classed as saheeh by al-Albaani).
The evidence that the marriage of a Muslim woman to a kaafir man is invalid is very clear. This is one of the matters on which there is unanimous agreement among all the scholars of the ummah. Allaah says (interpretation of the meaning):
“O you who believe! When believing women come to you as emigrants, examine them; Allaah knows best as to their Faith, then if you ascertain that they are true believers send them not back to the disbelievers. They are not lawful (wives) for the disbelievers nor are the disbelievers lawful (husbands) for them. But give them (disbelievers) that (amount of money) which they have spent (as their Mahr) to them. And there will be no sin on you to marry them if you have paid their Mahr to them. Likewise hold not the disbelieving women as wives…”
“And do not marry Al-Mushrikaat (idolatresses) till they believe (worship Allaah Alone). And indeed a slave woman who believes is better than a (free) Mushrikah (idolatress), even though she pleases you. And give not (your daughters) in marriage to Al‑Mushrikoon till they believe (in Allaah Alone) and verily, a believing slave is better than a (free) Mushrik (idolater), even though he pleases you. Those (Al-Mushrikoon) invite you to the Fire, but Allaah invites (you) to Paradise and forgiveness by His Leave, and makes His Ayaat (proofs, evidences, verses, lessons, signs, revelations, etc.) clear to mankind that they may remember”
What your husband says and attributes to Islam is definitely false. Islam is not only for the time when the Prophet (peace and blessings of Allaah be upon him) was sent, rather he (peace and blessings of Allaah be upon him) was sent to all of mankind until the Hour begins. Allaah says (interpretation of the meaning):
“And We have not sent you (O Muhammad) except as a giver of glad tidings and a warner to all mankind, but most of men know not”
“Say (O Muhammad): “O mankind! Verily, I am sent to you all as the Messenger of Allaah — to Whom belongs the dominion of the heavens and the earth. Laa ilaaha illa Huwa (none has the right to be worshipped but He). It is He Who gives life and causes death. So believe in Allaah and His Messenger (Muhammad), the Prophet who can neither read nor write (i.e. Muhammad), who believes in Allaah and His Words [(this Qur’aan), the Tawraat (Torah) and the Injeel (Gospel) and also Allaah’s Word: “Be!” — and he was, i.e. ‘Eesa (Jesus) son of Maryam (Mary)], and follow him so that you may be guided”
It was narrated that Jaabir ibn ‘Abd-Allaah (may Allaah be pleased with him) said: The Messenger of Allaah (peace and blessings of Allaah be upon him) said: “I have been given five things which were not given to any of the Prophets before me: I have been supported with fear to a distance of a month’s travelling; the earth has been made a place of prostration and a means of purification for me, so that wherever a man of my ummah is when the time for prayer comes, he may pray; war booty has been made permissible for me; the (previous) Prophets were sent only to their own people but I have been sent to all of mankind; and I have been given the power of intercession.” Narrated by al-Bukhaari, 438; Muslim, 432.
Your husband’s belief that “the important thing is our hearts should be clean. It doesn’t matter if we drink or gamble” is a false notion. If the heart is clean and pure then the effects of that should be seen in a person’s outward actions. Outward righteousness is a sign of inward righteousness, and outward corruption is a sign of inward corruption. How can his heart be clean if he drinks alcohol or gambles or commits immoral actions? This is impossible.
It was narrated that al-Nu’maan ibn Basheer said: I heard the Messenger of Allaah (peace and blessings of Allaah be upon him) say: “What is permitted is obvious and what is forbidden is obvious, and between them are doubtful matters about which not many people know. Thus he who avoids doubtful matters clears himself with regard to his religious commitment and his honour, but he who falls into doubtful matters falls into that which is forbidden, like a shepherd who pastures around a sanctuary, all but grazing therein. Truly every king has a sanctuary and Allaah’s sanctuary is His prohibitions. And in the body there is a piece of flesh which, if it be whole, all the body is whole, and if it is diseased, all the body is diseased: it is the heart.” Narrated by al-Bukhaari, 52; Muslim, 1599.
Conclusion: This marriage of yours is invalid and it is not permissible to allow him to be intimate with you unless he comes back to Islam and enters the faith by pronouncing the Shahaadatayn and following the rulings of Islam. If he does not do that then the marriage must be annulled in a sharee’ah court. If you cannot do that or if there is no sharee’ah court where you live, then you should ask him for a divorce. If he refuses then you should divorce him by means of khula’, returning his mahr or more or less so that you can be separated.
And Allaah knows best.