The colors are derived from the “spectrum” colors of the active theme. As groups are added, it checks to see which colors are unused and the next in the list gets picked from a static palette sequence.
As of today, users cannot manually select a specific color from this palette but it is something we have considered as a possible future enhancement.
The closest option is that if you have your tabs grouped-by-type, you can toggle that setting off and on again to have colors re-assigned but no specific manual color choice.