بسم الله الرحمن الرحيم
(Fatwa: 332/321/N=04/1436)
You were right that to leave the kissing of Hajr-e-Aswad (the Black Stone) in order to avoid hurting people. It is sunnah to kiss Hajr-e-Aswad when it is possible and one has access to it, whereas it is mandatory for Muslims to avoid harming people. It is hoped that you would earn reward on your good intention.
Allah knows Best!
Darul Ifta,
Darul Uloom Deoband
