Csep-561-Reading-1B
End-to-end arguments in system design
The purpose of this paper is to clearly define the end-to-end argument which favors implementing functions at the end of a network system (i.e., at or near the application level) rather than at the lower level (i.e. in the communication system).
Exhibit A: Careful File Transfer
Consider the transfer of a file from Computer A to Computer B, over a communication network. Failure of this transfer can occur on either of the computers' disk systems (e.g. hardware fault, buffering error) or within the communication system (e.g. lost/duplicated packets).
One solution would be a hash/checksum that could be computed by each computer on the file contents, to ensure correctness.
What level of correctness can be built into the communication system? At the most, we can achieve reliable data transmission. But since errors could still happen on the computers A and B, the checksum would still be necessary. So, at the most, implementing this function at the communication level could result in a lower failure-and-retry rate.
When to implement at the low level?
That's ultimately the case being made here: implementing functions at the lower level should be considered a trade-off of performance, rather than a requirement for correctness. And, its a tricky trade-off, with the cons being:
- other applications using the subsystem may not need the added functionality, yet they would pay for the overhead
- the subsystem often does not have as much information as the application, and hence might perform the function less efficiently.
Application Nuances
The context and requirements of the application are very important when considering low-vs-high functions. For example, live telephone conversions would suffer dramatically from low-level systems enforcing bit-perfect communication. The delays of resending damaged packets would be too jarring, whereas the natural redundancy of spoken language and the ability for the receiver to ask the sender to repeat lead to a setting where a damaged packet or two has little consequence.
But, change the setting ever so slightly: sending a recorded message to an answering machine. Now, such bit-perfect communications would have lower cost and higher value.
Thus, the end-to-end argument is not a strict rule, but rather a guideline.