In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of “quality” from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model’s output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.
10% 4chan
why didn’t they just say 0.4chan and be done with it?
Underrated comment.
Seems pretty rated to me
They taught it toxicity so it knows what they mean by “don’t be toxic”. It’s only a shame so few flesh and blood models take the same lesson away from it.
I know everyone on Lemmy hates LLMs, but this is really interesting
I don’t dislike LLMs, I dislike people who treat them as anything more than an advanced search engine and stupidly give them all their confidential data. Seen it happen too much at work.
I dislike that people are relying on them to do all their thinking for them while also being incredibly interested in the tech behind them.
I recently realized it’s a non-issue. The people doing this have already been looking for decades to find new ways to rot their minds. LLMs are just the latest in a long line of tools that help them tune out.
I’ve said this a few times in a different way and I always get downvoted. The fact is that the people who will use the LLMs to think for them, were not gonna think a lot in the first place.
This is a “guns don’t kill people - people kill people” kind of scenario.
As a standalone thing, LLMs are awesome.
What sucks is greedy people using them for the wrong reasons.
It’s like robots. Playing with robots are awesome. Firing 1,000 people and replacing them with robots - and not sharing the benefits with the community sucks.
As a standalone thing, LLMs are awesome.
They really aren’t though and that is half the problem. Everyone pretends they are awesome when the results are unusable garbage 80% of the time which makes them unusable for 99% of practical applications.
Those numbers are baseless exaggerations. There are plenty of tasks which they solve perfectly, today. It’s just that a bunch of dicks operate them, and the cost of operating them are way too high.
Also:
- environmental impact of AI
- unethical acquisition of training data
- dichotomy of how conservative politics treat AI company and private copyright law
- “undress AI” and deepfakes
It’s not that they’re not useful, that’s just nonsense.
There are plenty of tasks which they solve perfectly, today.
Name a single task you would trust an LLM on solving for you that you feel confident would be correct without checking the output. Because that is my definition of perfectly and AI falls very, very far short of that.
Who says you can’t check their outputs? It’s much faster to e. g. read a generated text than to write everything yourself. Same applies to translations, they’ve been excellent for quite a while now.
Business communication can be handled effortlessly by AI. Of course you read the result before you send it out, but that takes an order of a magnitude less time than formulating and typing all those meaningless sentences.
And honestly, that’s a perfect use case for AI. I wouldn’t compose a love letter to my family using AI, but a pamphlet, feature description, sales pitch, any bullshit presentation deck? You bet AI excels at those.
Same applies to content summaries that help augment search indices. Finding a large number of content candidates (e. g. videos) and have AI summarize the contents of said videos to narrow down the search is helpful and works today.
I’m not looking for AGI. I’m looking for tools to make my life easier, but in an ethical manner that doesn’t advance the destruction of the planet at an exponential rate, just for some tech bro to jerk it and buy another yacht.
You can make a generic fill in the blanks for all of those like I do and just change the key terminology for each scenario. LLMs are competing with search and replace?
I think this may be a skill issue on your part.
I wish they would tone down the crusade. This is some of the most interesting technology to come out in decades.
And I wish they would tone down the hype. Maybe we can meet in the middle?
Well, I do wish they would promote the actual use and limitations of AI and stop making up crap and overselling the use cases. I use ChatGPT at work all the time as a start for research, but if I took any of it as being reliable info to run with I would be in grave trouble. It is a great tool that has saved me much time because I know how far to trust it and how to use it. The progress is very impressive as I’ve been using AI art services for years, and the difference between the random blobs from back then and the great stuff it can generate now is pretty stark. Same thing with the LLMs. I’ve been using ChatGPT since it showed up and it has improved greatly since then. Before all this I talked to people who were using AI training on various picture recognition projects where getting data from other sensors was not practical. … Overall AI is pretty exciting, but the non-stop hype and hate headlines is doing nobody any favors.
It’s extremely useful for many things, if you know how to use it, and it’s annoying and useless for many others, which is what they fixate on and keep-jerk react to
Headlines should not say “scientists,” they should name the institution. (Harvard in this case.)
Headlines should not say “Harvard”, they should name the researchers. (Rachel Greene in this case.)
I don’t know why I had to write this.
Who’s Rachel Greene? But we all know Harvard and have an idea of their respectability. Name of the researcher if not well-known should be in the body instead.
“Harvard scientist Rachel Greene”
Everyone’s happy
Interesting - I can sort of intuit why it might help. Feeding the model bad data and instructing training it to identify it as such would be advantageous compared to being entirely unaware of it.
bad data
Can you define this? The authors/grifters call it “toxic data” but never define that either.
It’s a pretty simple concept. Train any kind of model on only “good” data, and it fails to distinguish between that data and bad data.
Take image recognition. Feed it hundreds of images of an orange and ask it to find the orange. After training, it will be very good at finding that orange.
Then add a picture of a Pomeranian dog in there, and watch as the model confidently marks it as an orange.
The model should have been trained on lots of images that don’t feature what you want it to output as well, so it knows to distinguish that.
I’m reminded of an early model that was trained to find if tanks were hiding pictures of forests / jungles. Was doing great with the training data then was given new images and seemed to be guessing wildly.
Turns out it in the training data all the pictures with tanks were taken on cloudy days.