We know that one must fill up the gap in between the rows in the masjid and that it is not permissible to leave an empty place. So in a multi-storied masjid, if one floor is not filled totally, how could it be permissible to pray on the second or third floor? May Allah reward you with Paradise.

All perfect praise be to Allah, the Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad is His slave and Messenger.

Completing the rows is a Sunnah and not obligatory according to the opinion of the majority of the scholars. The Sunnah is to complete the first row and then the rows behind it consecutively. Hence, the rows of the first floor should be completed first, and then after its completion, those who come after should go to the upper floor, but if they pray on the upper floor while there are still places on the lower floor, their prayer is valid.

Ibn ‘Uthaymeen said, "As for the people being led in prayer above the imaam (on an upper floor), then this is an issue of difference of opinion, although many scholars restrict this issue to when the imaam prays on his own on the lower floor while all the rest of the congregation pray on the upper floor. This is what is disliked, but if there is someone with the imaam, then there is no dislike regarding the issue of the people led in prayer being on the upper floor and the imaam on the lower floor."

However, this is contrary to the Sunnah, because the Sunnah is to complete the first row and then the second row and so on.

Shaykh Ibn ‘Uthaymeen was asked, "What is the ruling on praying on the second floor on the roof of the mosque while there is still space on the first floor, whether this is in the Sacred Mosque (in Makkah) or in any other mosques?"

He replied, "Praying on the second floor of the mosque is permissible if there is someone with him in his place; meaning that he is not alone in the row; but it is better for him to be with the people in their place (on the first floor); because if he is with the people in their place, then they will be closer to the imaam, and it is better to be closer to the imaam."

Allah knows best.