Common sense of the Nephilim return
In (Gen.6:4) we can see how the return of the Nephilim came to be. Basic demonic manifestation of these angelic beings called the sons of God is also stated in Job 1:6 /2:1 & 38:7.
Here is where the common sense comes into play. If this happened before the Flood, and we know all angelic beings are spirit, what would make any of us think these demonic angelic spirits could be destroyed by a flood? Their human offspring most certainly, but a spirit?
So in Numbers and Deuteronomy, when all of a sudden these human Giants start to appear again, all we are seeing is Gen. 6:4 happening again, only this time after the flood.
That should not surprise anyone. Its just simple common sense. I don't see a mystery here.