This principle actually has its roots in computer science & programming. The basic idea is to make each module focused and relatively independent, so that changes to one module don't ripple through the rest of the software. You'd like to have it so you can modify one module without having to go through and modify the rest of the code. This guy did a nice writeup about the concept, if you want more detail.
In a hardware context, this translates to standardized interfaces & outputs - sort of a black-box approach to modular design. In a good design, you should be able to pull out one part and replace it with something else with minimal (note: not necessarily zero) design implications for the rest of the system.
This approach takes effort and planning, but it simplifies the overall design, simplifies testing and simplifies upgrades.