Upgrade or new bike

Hi All, Last summer I got back into biking and upgraded my seriously old flat bar Trek to a BMC Roadmachine X. I upgraded the wheels to Fulcrum Quattro Racing Carbon Disc (not the best but I got a great deal on it), the saddle and changed the SRAM 1x gearing to 44T/10-42T. I like the 1x but from time to time, it is a bit "gappy".

All is well until now I see the Specialized Roubaix with Future Shock 2.0...

So, the question is, should I upgrade my Roadmachine X further with RedShift stem & seat post for more compliance and change out the SRAM 1x to Ultegra Di2 or should I buy an S-Works Roubaix frameset, Specialized handlebars, Ultegra Di2 and bring over my saddle, wheels & pedals from the Roadmachine X?

Moving to S-Works means more money for the frame, is it worth it? Is the S-Works Carbon frame that much better than my BMC Roadmachine X Alu frame? I don't race or group ride but I do like speed and long-distance rides.

Thanks for any suggestions.