According to Hanafi Fiqh, the time of Asr starts after two mithl. However it starts after first mithl as per Sahibain (Imam Abu Yusuf and Imam Muhammad) and as per Imam Shafi’i etc. But you should offer salah as per the preferred opinion i.e. after two mithl. It is mentioned in Ahsan Al Fatawa that it is preferable to offer Asr salah after two mithl though jamah is missed. (Ahsan al-Fatawa: vol. 2, p. 145)
If there are more than one Hanafi followers then you should offer Asr salah with jamah as per the Hanafi timing in any suitable place.
Allah knows Best!
Darul Uloom Deoband