Five minutes could save you $35 or more.

It pays to call your credit card company if they’re trying to hit you with a fee: often they’ll back down. That’s according to a new survey out Monday from CreditCards.com, which found that 87% of people with a credit card got a late fee waived just by calling and asking. What’s more, eight in ten cardholders got the annual fee waived or lowered and 69% got a lower interest rate when they simply called and asked.

And yet, only about one in four consumers actually pick up the phone to tackle these fees, the survey found. Big mistake: Late fees tend to range from about $15 – $38; annual fees average roughly $17 but can easily top $200; and interest rates hover around 15%, which on the average card with a balance can set you back $2500 in a year.

“People have far more power with their credit card company than they realize. Competition among card issuers is incredibly high these days and customer retention is a priority,” says Matt Schulz, CreditCards.com’s senior industry analyst. “Don’t be afraid to ask for fees to be waived or higher credit limits because, quite often, you’ll actually get it.”

What’s more, calling your credit card company typically isn’t as irritating and time consuming as a phone call to the cable company or airline. “When it comes to getting a late payment waived, it really could take as little as 5 minutes to call,” says Schulz. Lowering your interest rate or waiving an annual fee may take longer, but because the savings can top $100 and sometimes $1000, it may be worth it.

“Ask politely,” Schulz says. Experts say that it may help to point out that you’re a long-time customer who tends to pay on time. Plus, come armed with other credit card offers, so that the company knows you may be serious about leaving.