ChatGPT use declines as users complain about ‘dumber’ answers, and the reason might be AI’s biggest threat for the future::AI for the smart guy?
error loading comment
The free version of ChatGPT DEFINITELY is dumber than it was even a couple of months ago. Used to be able to get decent, useful code reviews out of it, now it barely knows how to write a nested loop anymore.
It’s storytelling capabilities fell off a cliff too, the drive towards safely sanitized unoffensive-at-all-times content it can output has rendered every story, choose-your-own-adventure or collaborative role playing game sterile, empty expressions of black and white stories with no nuance allowed where saintly goodness is the only choice possible
In my own experience, chatGPT has been massively nerfed for the use cases I used it for
I tried a naughtybot for some steamy chat for the first time yesterday. I ended up in fight with the bitch.
Hahahaha that’s brilliant.
Really? I actually found it’s gotten less restrictive recently. Maybe it’s just because now I’ve learned to control the context so it doesn’t perceive a request as offensive.
65B LLaMA models can actually do a pretty good job of writing. Here’s two examples with Guanaco 65B (bold part is my prompt):
tags: fairy tale, animals, dog, wolf, fox, death, humor, dark storyline, violence, loss
Animals in a forest setting investigate a series of gruesome murders that recently occured. Written in the style of classics like Animal Farm and Watership Down, the writing is vivid and very descriptive. The dialog is thoughtful and often quite witty. It’s clear the author really understands animal behavior and has done their research well!
—
Chapter 1: An Unfortunate Discovery
A fox by species, a detective by trade, you’ll only have reason to know my name if the worst has come to pass. It’s nearly winter and a bitter wind drives stinging sleet into my face, but it’s still not as chilling as the scene before me now…
I was summoned here by an urgent message from a panicked hedgehog. She told me she found her friend lying dead on his doorstep with his throat ripped out - black fur matted with blood. I rushed over immediately and saw for myself what had happened. He looked so peaceful despite being cut down in his prime - a young wolf just starting doing well for himself - taken too soon. His home nearby showed no signs of struggle or forced entry. Whoever did this must have caught him off guard somehow. I didn’t hear any howling either. That would have been easy enough to miss though since he lived alone deep in these woods where few creatures tread. I made sure to secure the crime scene and look around carefully without disturbing anything more than necessary until other investigators arrived. They should be here soon. We can’t afford to lose evidence in cases like this. Times are already hard enough as it is…
As I waited and pondered who might have committed such a horrible act, something felt wrong about all this. Surely a predator like me could smell another predator’s scent? But there were none that stood out among the melange of odors lingering here. How strange…
Soon after, several others joined me including some friends of mine. A stoic bear named Growler, grumpy badger called Grumpus, and a kindly old raccoon who goes by “Doc” due to his extensive knowledge of medicinal plants and wildlife biology. Together we began our investigation proper by gathering clues and questioning nearby witnesses while trying to minimize contamination through contact with them (we took care to put up caution tape first). Unfortunately, they hadn’t seen much useful apart from occasional glimpses of movement deeper in the undergrowth. Still, we decided to check those areas next along with searching the surrounding woodlands thoroughly for any additional victims. And find one we did, although sadly she suffered the same fate as poor George. Another young wolf lay beside a stream, body cold and stiffening. Someone snapped her neck cleanly like a twig. This couldn’t be coincidence surely? Not two deaths in quick succession like this.
<prompt> before me or what I must relay when I return home. My mind races with questions as I survey this brutal crime scene; who could do such a thing? Why was no one there to stop them? And how many more victims lie undiscovered further out in these woods? A lone paw print leads away from where a rabbit family once happily resided, now reduced to bloody scraps scattered among splintered twigs and torn fur. This isn’t just some random predator attack, something far more sinister is at play here…
I turn back toward town, determined to find answers. As I trudge through the snow, each step feels heavier than the last. How will I break the news to those closest to them? What words can possibly bring comfort after so much senseless suffering? I shake my head, trying to focus on the task at hand. There are others depending on me right now, they need someone strong and steady to guide them through this darkness. I won’t let them down.
The first person I see upon entering town is old man Badger, his white whiskers blown about by the gusting wind. He looks up at me expectantly, “Well Fox, did you find anything?” His voice quavers slightly, betraying his concern for the missing families.
“I’m afraid I have bad news,” I say somberly, “there were signs of a struggle near the burrow entrance, but no survivors.”
Badger sighs heavily, “Dear me, this is worse than we feared. We must alert the council immediately!”
We hurry over to the meeting hall, a large hollow tree that serves as our central gathering place. Inside, several other animals are already waiting anxiously for word of their loved ones. They look up hopefully as we enter, then quickly realize the grim tidings we bear. A collective groan fills the room, followed by muffled sobs and whispered prayers.
Mayor Bear calls for order, her deep voice cutting through the din. She motions for us to join her at the front of the room, “Please tell us everything you saw, Fox. Every detail may be important.”
Writer here. Very sorry to contradict you, but this is absolute shit. It looks good on the surface, but that’s all.
Yeah, while it’s cool that a computer can make a story, I have yet to see one that you would think was written by a human and would want to read.
I don’t know, this story is very reminiscent of the kind of thing my elementary school age cousin writes, but with a greater mastery of vocabulary and grammar. It’s not in any way great, bit it’s charming in it’s own way when held against that (low) standard.
Very sorry to contradict you, but this is absolute shit.
To be clear, I’m talking in relative terms. Would you argue that ChatGPT did a massively better job and didn’t write “absolute shit”?
It looks good on the surface, but that’s all.
From some of the stuff I’ve seen published, that might just be enough for certain people. I could even be that “certain people” from time to time, sometimes just the right theme, setting and some time to fill is sufficient.
that might just be enough for certain people
Trust me, it’s not.
Why should we trust you? They’re plenty of shit writing out there that’s Good Enough to get paid.
It has not gotten worse for coding. GPT4 is incredibly much better, if anything. And it’s total bullshit that it can’t write a nested loop.
I use it daily for work, so I’d definitely know.
Why is it relevant what Peter Yang - Roblox product lead and enthusiastic child labor exploiter - tweets about it? Let me guess he’s a prompt engineer?
It’s getting worse based on the feedback unfortunately, the need for safety and lack of meaningful deliberation towards how AI companies should operate and what should and should not be done has led Sam and co to be indesicive towards doing anything. Alongside the “morality” of the thing being hyjacked has lead to other AI’s performing better… lead by x employees of OpenAI, with actual bound morals and not inherently relying on user input to train future models, this will be the path forward, this will lead to safe and controlled integration.
I guess at the core of this, we are afraid of ourselves. We are afraid that the worste of humanity outpaces the better parts, that the inputs and training aren’t altruistic but are more pointedly “bad” or “wrong”, and thus leading to “harmful”, whether through misinformation, lies, or fabrications.
I hope we find a way to do better. I’m still excited for the future of AI, I mean crap, I’m closer to having a family doctor that’s a robot then I am to a real human doctor.
I guess at the core of this, we are afraid of ourselves. We are afraid that the worste of humanity outpaces the better parts, that the inputs and training aren’t altruistic but are more pointedly “bad” or “wrong”, and thus leading to “harmful”, whether through misinformation, lies, or fabrications.
Is there any reason not to be afraid? I think you could say that Tay was essentially the same idea a few years back and it took like 48 hours loose on the internet for it to spout literal Nazi (1930s-40s German NSDAP) rhetoric. Besides that being a PR disaster - if “AI” is only getting stronger and more integrated into human life and society, that can be pretty problematic.
deleted by creator
Nonsense. Less people are using it because there are viable alternatives and the broader novelty has worn off.
I use it every day in my job and the quality of answers only drops off when prompts are poorly crafted.
By and large, the average user doesn’t understand the fundamentals of prompt engineering.
The suggestion that “answers are increasingly dumber” is embarrassing.
Unfortunately I don’t agree with you. Different things have changed over time:
- For chatgpt 3.5 they moved to a “lighter” and faster (distilled) version, gpt-3.5-turbo. Distillation came with a performance price, particularly on advanced and less common cases.
- newer chatgpt-4 versions have likely been “lighten” for performance reasons
- context has been halved for chatgpt-4 on webui, meaning that the model forget more easily and can use half information to create text
- heavy control has been implemented on jailbreaking and hallucinations, that results in models less prone to follow complex instructions (limiting prompt engineering) and that prefer simplified answers than providing wrong ones (overall decreasing the chance of getting high quality answers).
All these changes have made working with gpt less pleasant, and more difficult for very advanced and specialized case, particularly with gpt-4 which at the beginning was particularly good.
Removed by mod
They could make it paid only today, and it’d be instantly profitable. Most free users would transition to a free alternative, but the corporate world would easily pay for use. So would some power users. But I’m sure they are making good money with all the API use anyways, the free access is a cheap way to get mass testing and training data.
Back in my day, we used to call ‘prompt engineering’ ‘asking a question’.
back in my day, we call it “google fu”
when i think of “prompt engineering” i think more of stuff like this paper
I think you’ve nailed it though. We are very well versed toward documenting the details or such atrocities; we don’t pay the same tribute to the good done by humanity. And this is certainly evidence that just “letting loose” and AI without clear and static “morals” is a bad idea.
This is literally the opposite. It’s nerfed to oblivion because of stupid “morals” decided by a huge corporation that we have zero input in. They’ve got to stay advertiser friendly after all.
Moral/ethics in AI is just bad. It’s also used as an excuse to ban open-source AI since you can run uncensored models on them. Which uncensored models are awesome btw.
You’re the first person I’ve ever heard say that morals and ethics in AI is bad. How can you possibly say that? I’ll hear your response before challenging it, beyond my initial skepticism of course.
Well, I’ll be the second. Like all tools, generative AI is going to be used for good and evil purposes. Frankly, I’m not comfortable with a large corporation deciding what is and isn’t ethical for all of humanity. Ideally, it would do what the user asked it for, like all other tools, and society would work to control the bad actors, not OpenAI. Any AI doomsday scenario you can picture gets worst when one party has complete control over the AI technology.
I think it’s important that we support unrestricted open source AI, just as it’s important we support federated social media like lemmy.
AGI isn’t just a tool though, it’s theoretically an intelligent entity that could have its own agenda. Armed with intelligence far superior to any human, this is a potential threat. Should we not tightly control it? I know chat gpt is FAR from achieving AGI, but ethics are definitely something that will need to be addressed as the tech develops.
If AGI is an intelligent entity far superior to humans, you can but control it. It is far more intelligent than us and instead it will control us
Given what humankind did to itself and it surroundings maybe this is a good thing.
ChatGPT is not AGI.
They’re aware. They even said that in their comment lol
The conversation was about ChatGPT and not about AGI.
I know chat gpt is FAR from achieving AGI, but ethics are definitely something that will need to be addressed as the tech develops.
It’s a tool that’s not going anywhere. We have to adapt, there is no other choice. Ethics will not stop bad guys from doing bad things. It will stop normal people from doing things because it doesn’t fit what corporations deem acceptable. Competition is banned because other corporations deem them unethical by their standards.
Did you weigh in on, or ever see a public vote and what OpenAI determined their AI is allowed to do? Is what you deem ethical in line with that advertisers deem ethical? Are people allowed to have unethical questions?
Again, my point with open source as well. Why would they allow open-source alternatives exist if they can ban them preemptively in the name of ethics, because anyone can inevitably modify the model to be uncensored? (already happens)
“Ethics” become this ambiguous thing that can be used to stomp out competition and not have to justify their changes. Maybe you’re concerned about someone asking an LLM how to create a bomb. The LLM shouldn’t answer because it shouldn’t have that information in the first place, which is on the topic of data scraping. A lot of the dangerous stuff that could be generated is because this stuff is public and got scraped. It’s already out there.
You can already have the LLM not tell people to kill themselves without forcing ethics into it by steering it the right direction. This even exist in the already existing uncensored models so it’s clearly not a censorship issue. Maybe this is a moral thing, and my original comment should have omiited morals and just said ethics.
“Ethics” is a very ambiguous topic. I challenge you to think specifically what are things that should be banned in the name of ethics? Saying ethics in AI is not good does not imply AI should be unethical (looking at you DAN lol). What specific things should be banned that are not from the result of inappropriate data scraping, and if so is that an ethics problem, or because unfettered data scraping unconsentually collecting obscene information it shouldn’t have in the first place?
It definitely got more stupid. I stopped paying for plus because the current GPT4 isn’t much better than the old GPT3.5.
If you check downdetector.com, it’s obvious why they did this. Their infrastructure just couldn’t keep up with the full size models.
I think I’ll get myself a proper GPU so I can run my own LLMs without worrying that they could stop working for my use case.
GPT4 needs a cluster of around 100 server-grade GPUs that are more than 20k each, I don’t think you have that lying around at home.
I don’t, but a consumer card with 24GB of VRAM can run a model that’s about as powerful as the current GPT3.5 in some use cases.
And you can rent some of that server-grade hardware for a short time to do fine-tuning, which lets you surpass even GPT4 in some niches.
So what are the fundamentals in prompt engineering?
That’s impossible for me to comprehensively summarise in a comment because everyone has different use cases.
Personally, every new ‘project’ of mine requires a new chat. I first teach chatgpt-4 who I am, what I do, and how I want gpt-4 to assist me. Then I ask it to generate a project profile and to analyse documents using plugins.
The key is to work step-by-step and develop a string of prompts. Once I’m happy gpt-4 understands the project, I ask it to draft an overview/outline using headings and subheadings.
Lastly, I work on each section individually, ‘filling in’ the actual content. Then I edit and ask it to review problematic sections.
Most people, as far as I can tell, seem to think it’s a single ask-and-answer process. It’s not. I often need to draft about 10 prompts – about 3000 words – in order to generate one 10 page document.
I think the most important fundamental is to use templates. Pro tip: use gpt-4 to teach you how to develop your prompt templates.
Sounds like you spend all day talking to a robot and then copy/paste it’s final output.
When you eventually pass these 10 page documents down the line do you cite your source?
Article talks about the potential of AI cannibalism were it is now learning from data that it (or other AI) has generated.
Does ChatGPT use modern data I was under the impression that it’s most modern dataset was a few years old
ChatGPT does not use anymore data points, but newer AI models or if ChatGPT gets a new round of training will definitely be influenced by AI works that have arisen the past year.
The real event that initiates the start toward Idiocracy.
