Was just reading this piece by Tim Cochran of ThoughtWorks -- it's interesting, although definitely uses its fair share of buzzwords. One thing that rings extremely true is the idea of "quick feedback loops".
As a developer, you want to be able to make changes and test that they work in as little time as possible. When builds take a long time, even 15 seconds, you lose focus: you check slack, you switch terminals to check on something else, you open your phone.
It can seem like a waste of time to spend many hours or even weeks optimizing a 15-second build to take 3 seconds (after all, it's only a difference of 12 seconds), but you're not just saving 12 seconds on every build -- you're saving your flow. Every time you make a change, you stay on task. You may even find yourself making more changes, testing more cases, and writing better code.
Optimize your build times!