Assigning Audio Devices to Programs

On my Windows PC I have multiple audio devices. There is a 2.1 speaker system attached via USB and an analog headphone jack on the front. When I am not alone, I attach the headphones and want it to be used. Otherweise I remove the headphones and want to use the speakers. For some reason Windows has the super obnoxious limitation that audio sinks cannot be changed after the application has launched. There is a settings menu where I can apparently set the source and sink for separately for every application. But it does not seem to have any effect when the application is already running!

There is the following settings menu, which I have tried to change the audio sink after starting a game that takes like three minutes to start. The game was listed there, but it did not have any effect. Also changing the default audio device did not change it for the already running game.

On Linux with Pulse Audio I have this super handy dashboard (pavucontrol) where I can see all the streams and assign them to an audio sink. The change will happen immediately. This way I can to things like having a voice call using the headphones but also play music via a USB sound system.

I really do not understand why this is not possible with Windows. It is super annoying having to close the game again and restart (wasting another three minutes) just because I forgot to plug or unplug my headphones before starting it.