If you have doubt to feel downgrade or not I must have felt big downgrade going from M6 to M4 right? But I don't have that feeling.
IMO the current 6 series is one of the most beautiful BMW even made and M6 is the top of the line with amazing power and all sporty-luxurious features but M4 is a different animal. It's lighter and smaller and more nimble. It's a fun car and to me not comparable to M6 or even 6 series... it's an sport car ... let's say this way... When I jumped to M6 and compared it with my 650i, I felt the sport+ mode in 650i is still more comfortable than comfort mode in M6. And now I jumped from M6 to M4, I feel the sport+ in M6 is more comfortable than comfort mode in M4... Did it help you?
All I could say is to enjoy the ride and life and don't worry about those feelings
