They aren't out of context, and you have just said the same thing. Data processing can help in removing noise, but it can't help in creating information or extracting information that wasn't there in the first place. In fact – again as you said – it can end up destroying part of the original information.

LLMs extract word correlations from textual data. Already in this process they are losing information, since they can't extract correlations beyond a certain (yet large) length, and don't extract correlations at shorter lengths. And in creating output they insert spurious correlations that replace (destroy) some of the original ones. This output will contain even less information than the original training data. So a new LLM trained with such an output will give back even less.