Cycling clothing

I have been commuting my short distance to and from work for last 18months or so most days and haven't had much problem with my adapted cobbled up old bike and similar clothing.
Recently I've been getting in to using my bike recreationally for the occasional longer weekend ride and have been enjoying them as part of my destress time (my other half and I used to do a lot of hill walking but our recent arrival means long winter fell walks aren't practical now).
As I'm spending more time on the bike Ive treated myself to a proper winter cycling jersey which I used for the first time today.
I have until now being using my old gym kit, and hill walking gear etc cobbled up to keep me warm and its been working fine but today was so much easier on the bike, I got up to speed faster, reached gear changes quicker and felt like I was coasting at speed more.
Can a tighter fitting Jersey really make that much difference (was the only change in my kit) or was it all psychological? I will have to test it on my actual commute to get a better feeling I suppose.