Limiting the access from external network can ensure that the access is secured.
Currently, there is no way to limit the usage of HubSpot App so that it can only be used if it's coming from specific IP Address.
However, HubSpot's Product Team is working on Limiting access Via IP Address feature