The only fix is probably replacement or having it recovered.
My concern over the years is….what causes this degradation? It can also be found on the top section of the driver’s door. Both locations are places where the driver rests their arm…which makes me assume that the material is being affected by something the driver has applied to their arm….like some type of lotion (hand/body/sun screen/etc).
After seeing these areas deform on some vehicles…anytime I’m not wearing something with long sleeves…I keep a hand towel folded up in the door’s storage been, then place the towel on the center armrest when I have bare skin…especially if its hot and I’m sweaty. I also clean these leather surface frequently…anytime I wash the vehicle…I take a dampened microfiber towel (also folded and stored in the door bin) and wipe down all the leather…especially the center armrest, steering wheel, and top of driver’s door…as well as the entire dash (since I have the leather dash option in both my BMWs).