I don't know if you've got the wrong end of the stick. I think if you deleted any reference to your home WiFi it would avoid any confusion because you're right the phone or controller connect to the WiFi on the drone directly.
The firmware update gets onto the phone via a WiFi with internet, but...