Why a tax rate can't be deleted
If the Delete option on a tax rate is greyed out or an "in use" warning appears, the rate is locked because it is referenced by other records — or, in one specific case, because it is the tax-exempt rate.
Once a tax rate has been applied to an invoice, estimate, or product, deleting it would alter the historical totals on those documents. The lock is permanent for the lifetime of the related documents, including cancelled or credited invoices.
The tax-exempt rate is also undeletable, regardless of usage. Sliptree uses it as a fallback whenever taxes are switched off entirely, so it is always kept around. Other built-in rates can be deleted when they are not in use.
Deleting the rate currently marked as the business default is allowed — the tax-exempt rate is promoted to default automatically, and a different rate can be marked as default afterwards.
What to do instead
An in-use rate can almost always be deactivated: toggle it to inactive and it stops appearing in selectors, but stays attached to its historical documents. See Managing tax rates.
When an official rate changes (a VAT rise, a new reduced category), create a new rate and update products to point at it rather than editing the existing rate. See Built-in vs custom tax rates for the reasoning.
Customer and product deletion follow different rules — see Bulk actions on customers and products.