Any single one of those things may seem harmless... "How can it be bad to add annotations to a web page?" The problem is that there are no technical limits to what patching can do. On a technical level, once you grant a third party patcher access to a message, they have complete control over it. They can change it to anything they please, with or without notifying the sender or the receiver of exactly what they are doing. (In the receiver-initiated cases, the patcher tells the receiver something about what they are doing, but there is no way for the receiver to be certain that the patcher is telling the whole truth, except through trust. To be fair, this is true of any software program.)
It is never a matter of "just adding annotation" or "just censoring certain scenes" or "just blocking advertisements". A patcher may claim that they are exercising restraint but at any time that restraint can end.
It is entirely unethical to ask a sender to send a message under those conditions. The value of the communication to the sender extends only so far as they have sufficient control over the message to do their best to ensure that their message expresses what they want it to express. When a third-party patcher is invited into the transaction against the sender's will, no matter what the patcher claims to be doing or even what the patcher actually is doing, the message ceases to be under the senders control and it becomes the patcher's message.
How can I claim it's actually becomes the patcher's message when it (usually) looks so much like the original message? Well, ask yourself who has control over the contents. For the purposes of concreteness let us talk about proxy-based annotation in particular for a moment, though the principle trivially extends to all other instances of patching. Support the web page author decides to add something to the page. Who controls whether that gets to the receiver if the receiver requests the page again? The proxy server can completely mask the new change out, in such a way that the receiver never notices. On the other hand, the author exerts no control over the proxy whatsoever; the author is probably not even aware of the proxy's existence. If the change gets through to the receiver, it is because the proxy graciously allowed the change through, not because the author made the change. There is nothing the author can do that the proxy can not veto, while the author has no say in what the proxy does.
Again, follow the effects. The patcher can effectively make any change they want. The patcher effectively has full control, even to explicitly contradict the author's wish. Therefore, the logical conclusion is that they do have full control. In terms of the effects, a patching system is equivalent from the receiver's point of view to the patcher actually changing the source code of the original message. If they actually did that in the real world, if ClearPlay actually went and modified the actual movie, if annotation servers actually hacked into web servers and changed the code for the page, if spam blocking programs actually hacked into other's computers and broke the mail sending facilities (and remember that very little spam comes directly from spammers nowadays, they relay from other people so you'd be hitting innocent victims, as tempting as such a drastic solution might sound), we would instantly know that something unethical has been done.
This is the "patch hole", a hole not just in the law but in our understanding of the world. Where we should see one unified effect, we see two sources and some technology for combination. We allow ourselves to be distracted by irrelevant technology behind the scenes and allow the technology to obscure the fact that accomplished in another manner (direct modification of the original source material) the action is obviously unethical.
Each of these patch-based technologies seems so innocent on first glance, but taken together, and taken to their logical conclusion, it paints the picture of a world where nobody can every have any assurance that a received message bears any resemblance to what they meant to say.
In each instance, there is a better way, or it's simply better not to do it. Any information a patch is adding can be communicated by other means; one need not mark up a web page to discuss the contents, one can go online to any number of bulletin boards dedicated to the task of communicating for that. Any patch that removes information, such as ClearPlay's product, should either simply not be done since removing information is a violent act that does unquantifiable and unqualifiable damage to a message, damage that nobody is fit to judge, or should be done with the explicit cooperation of the sender. Even if you aren't quite willing to believe right now that the costs are as high as I say they are, I hope you'll agree that the benefits are still so marginal and of such low quality compared to better solutions as to not be worth it.