Empirical studies on software refactoring are hard to find, but I believe refactoring follows the 80/20 rule - 80% of the possible benefit comes by applying just the first 20% of effort. Like most work in engineering and economics, the curve flattens under the pressure of diminishing returns. In this context I'm not thinking of individual refactoring operations (like extract method), but overarching efforts to improve software quality through rework, or by following a process like test-first design.
I think most of use could agree that the best place to be on this curve depends on context. Start-up software is different than complex software for a mature enterprise.
Taking context into account - do you feel like you operate in the sweet spot of the curve for your project?
My suspicion is that developers who read blogs feel like their project are always too far on the left hand side of the curve, but I wonder if the business owners feel the same way.