It was not good on behalf of your father to give his elder son alone in his lifetime and deprive the other children. However, now the ruling is that if your father gave the site in the complete possession of his elder son then he became its owner. Now the ruling of inheritance shall not be issued in it rather the elder son shall remain its owner. However the ruling of inheritance shall be applicable in the property which the father left behind at the time of his death. Thus your brother is correct. The property of your father shall be divided into eight shares out of which 2-2 shares shall go to each son and each daughter shall get 1-1 share.
Allah knows Best!
Darul Uloom Deoband