When you visited Makkah for three days and performed Umrah, it was valid. Hajj becomes fardh on one who has not yet performed his obligatory hajj and staying up to 9 Dhul Hijjah is possible for him without any difficulty. But since one may not opt to stay in Makkah himself, rather there are legal restrictions; therefore your Umrah was alright and the hajj-e-tamattu was also valid.
Allah knows Best!
Darul Uloom Deoband