In a case like this, the devil is all in the details. The Knicks needed J.R. Smith, J.R. Smith needed the Knicks, and this was about the only way either side could get a fair deal done.
Smith is one of the few NBA free agents that has Early Bird rights, which happens when a player either signs a two-year contract or, in Smith's case, inks two straight one-year contracts with the same team. How does that work? Via Larry Coon's salary-cap FAQ:
A team may use the Early Bird exception to re-sign its own free agent for up to 175 percent of his salary in the previous season (not over the maximum salary, of course) or 104.5 percent of the average salary in the previous season, whichever is greater.
Smith made $2.8 million last season, so in this case, 104.5 percent of the average salary is greater than 175 percent of his previous salary. The four-year deal is therefore essentially for 104.5 percent of the average salary. The $24.5 million number is an estimate, with the final figure to be released once the new CBA year begins on July 10.
The Early Bird exception was New York's only means to keep Smith. They'll be over the luxury tax next season, so they only get the mini mid-level exception starting at $3.183 million instead of the full mid-level exception starting at $5.16 million, or just under Smith's average annual salary. The Knicks may need that to keep Chris Copeland, and if not, they'll definitely need it to add other pieces to the roster. Having to use it to replace Smith would have been disastrous, as there's no way to find an equally-important player for half the price and still improve the team.
Smith, on the other hand, was backed into a corner when nobody seemed willing to meet his asking price, which reportedly approached $10 million. The Rockets and Mavericks considered it, but were unwilling to get serious until after Dwight Howard made his decision, according to Marc Berman of the New York Post. With no other offers for above the mid-level exception materializing, Smith had no choice but to take the best he could get from New York.
Therefore, both sides needed each other. I'd normally be wary of paying Smith guaranteed money for four years given his flaky history, but the Knicks had little choice. They already got Smith to agree to two straight one-year contracts; they weren't going to get him to agree to a pay cut again. Replacing Smith's role on the team given the limited salary-cap options would have been impossible. So, the Knicks made the only choice they could make, and now, they'll have to live with Smith's consistently inconsistent production that either makes the Knicks an elite team or shoots them in the foot.
You could absolutely blame the Knicks' poor cap management for putting them in this position in the first place, but that's a different evaluation. Given the unique circumstances, it's tough to blame the Knicks for this deal.