There is not currently a way to fully delete old upsells. The developers left this out due to not wanting historical data to be deleted along with deleted upsells.
The only thing you can do currently is keep the “Show Active Only” checked to not show the inactive upsells.