Your math is not wrong for the thought experiment that I was doing. The thought experiment was what if I got all of my 2% Neanderthal genes from a single ancestor? If that were the case, it would only be back to the late 1700s or early 1800s.

In terms of the closeness of Neanderthals, no, none of us have anywhere near a full-blooded Neanderthal that recently in our past. For all people with some non-African ancestry the foundational populations of the early hunter-gatherers, Anatolian farmers, and steppe horse people (and others - all Asians and Native North Americans) all have deep ancestors who mixed with Neanderthals about 40,000 years ago. Then those people spread out and carried their Neanderthal genes around with them. So, each of my 4 times great grandparents had about 2% Neanderthal, and that general percentage is conserved in each generation.

I think the book did go into details for how long it took to absorb the other populations after they were displaced. There are so many details in the book, and I just ended up with broad brush strokes. The takeaway of that in every case, the genetic history shows that every time we think of a people being "wiped out" by another people, that ends up being an oversimplification. The genetic history keeps showing a pattern of side-by-side living followed by a gradual merging. By gradual, I think they are thinking in the time scale of thousands of years, but I can check back in the book to verify.