Praise be to Allah.
Saying the Basmalah when reciting al-Faatihah or another soorah when praying is one of the Sunnahs of prayer, as is saying Ameen. If a person forgets either of them, he does not have to do the prostration of forgetfulness (sujood al-sahw). It is mustahabb for him to do it but if he does not, there is no sin on him and his prayer is valid.
Shaykh Ibn ‘Uthaymeen (may Allaah have mercy on him) said: If a person omits any of the mustahabb words or actions because he forgets, but he usually does them, then it is prescribed for him to prostrate (the prostration of forgetfulness) so as to make up for this shortcoming, which is falling short with regard to what is perfect, not falling short in what is obligatory. That is because of the general meaning of the hadeeth: “For every mistake (sahw) there are two prostrations.” In Saheeh Muslim it says: “If one of you forgets, let him prostrate twice.” This is general in meaning. But if he omits a Sunnah that he does not usually do, then it is not Sunnah for him to do the prostration of forgetfulness, because it did not occur to him to do it.
End quote from al-Sharh al-Mumti’ (3/333).
For information on the pillars of prayer, and it obligatory and Sunnah parts, please see the answer to question no. 65847.