I had a Mercedes 190E and it was very prone to this, after some investigation it turned out to be flat spots on the tyres (from locked up brakes)
The big clue in your post is that it is more noticeable on smooth road surfaces.
Modern cars which have ABS should be less prone to this but it has happened on my E class estate over the years - when I went to France the super smooth roads highlighted the issue big time!
Have you tried new tyres?
Pricey but a clean sheet start
That was my initial hopes but 2 sets of tyres later and still there