If you reject the provable existence of nonstandard numbers as a delusion then you reject arithmetic theory.
Nonstandard numbers are not "just a philosophical interpretation". They are provable from the compactness theorem, from the incompleteness theorems, and from ultraproducts. As Gödel pointed out, it requires mathematical intuition and special ability to perceive nonstandard numbers directly and to fully understand why they are there.
Victoria Gitman has an interesting lecture in which she describes the shape of the arithmetical multiverse:
She also points out that there exist indiscernible numbers:https://victoriagitman.github.io/talks/ ... metic.html
Order-wise, these models look like the natural numbers followed by densely many copies of the integers: N followed by Q-many copies of Z (see the slides for explanation).
This is actually nothing unusual. Through Richard's paradox (1905) undefinable numbers had already been discovered.In particular, a nonstandard model of arithmetic can have indiscernible numbers that share all the same properties. Indeed, the automorphism groups of nonstandard models have some remarkable properties themselves (for details, see [2]).
There are also ineffable numbers: https://en.wikipedia.org/wiki/Ineffable_cardinal
Because all of this is provable, it is mathematics and not mere "philosophical interpretation".