Fiddle leaf care

I just got a new fiddle leaf and it has seen some better days. Still healthy and sprouting at the top though but some leaves are pretty beat up.

Should I cut the leaves off at the base or just let them stay?