>the whole "swing wing" phenomenon the dumbest chapter in the history of aircraft design
Hmm, why do you think so?
Air resistance patterns are pretty different at different speed. For example, serious supersonic speed requires narrow, arrow-like design without large wings, because air travels much different than at transonic or subsonic speed. Transonic and "low" subsonic requires angled wing, and best performance on subsonic is achieved with straight wing.
You may see this in different designs, i.e. compare F-104 with F-4 or Sabre - they had different initial design goals and very different in appearance. And they all have flaws, for example, F-104 had pretty bad handling and crashed often. Good all-around wing design is very hard to achieve, especially in past, when no computer models exist (even F-117 was rugged-square-like not because they want this, but because they couldn't model radar deflection properly in 70s).
Another reason is lack computer-aided control - it is very hard to pilot a plane that has bad performance on specific speed. Especially on takeoff/landing. You want your fighter to 1) operate from non-high-class airports (they are rare) 2) don't crash often.
Even now most of modern fighters aren't too fit to constant supersonic flight, although they weren't made for this. But in 60s people really thought that dogfights are gone and planes will just shoot missiles at each other on 1.5M.
So, variable-sweep wing is pretty interesting and rational answer to all these problems. One big drawback is a large, complex and heavy mechanism that need to be operated even in flight, but it is still a solution.
They aren't user much because some problems are solved (computer models can fix aerodynamics), and change in plane usage tactics - no one cares much about high-altitude high-speed flying because you can be hit by SAM even there, it isn't 60s. Multipurpose (i.e. cost-efficient) planes now require good subsonic performance, not supersonic.