Note: This retrospective is on GPT-3.5, not any other OpenAI language model.
This is a retrospective on what it was like to use GPT-3.5 during its peak, why it caused so much frustration, and how that experience shaped my view of AI, tooling, and infrastructure.
How It Started
When I first discovered GPT-3.5, it felt revolutionary. An AI that could write, explain code, answer questions, and help with technical problems sounded incredible. I expected something that could assist with real-world projects.
Instead, I walked into the roughest era of modern AI.
Confidently Wrong Information
GPT-3.5 didn’t just make mistakes — it made them confidently.
- GPT-3.5 incorrectly claimed that MyOwnFreeHost does not have an API, even though an API has existed since January 2021.
- It falsely stated that the API was introduced later, despite documentation being available since early 2021.
- GPT-3.5 said Blooket did not have a game mode called “Crypto Hack,” even though it has existed since July 2021.
- It incorrectly claimed that “Crypto Hack” was introduced after its September 2021 knowledge cutoff.
- It demonstrated inconsistency by knowing about Blooket’s “Tower Defense” game mode but not “Crypto Hack,” despite both existing during the same period.
- It hallucinated broken URLs that led to 404 pages.
- It invented features, services, and product history details that were not true.
Worse, it would hide behind phrases like:
“As of my last knowledge update…”
“As an AI language model…”
Those phrases didn’t add caution — they added false confidence.
Made-Up Services and Categories
Some hallucinations were borderline absurd:
- Suggesting Freenom as a subdomain redirect service.
- Saying Freenom was an alternative to Freenom.
- Inventing fake services like “FreeNomains.”
- Recommending paid services like BeenVerified as “free.”
These weren’t edge cases — they happened constantly.
The Windows 3.1 “Alternate Solutions” Incident
One of the clearest examples of GPT-3.5 completely losing the plot happened when I asked about modern web browsing limitations on Windows 3.1.
Instead of explaining the real, well-known technical constraints, GPT-3.5 suggested a list of so-called “alternate solutions” that had nothing to do with the problem.
- It suggested emulating Windows 3.1 inside a modern web browser as a workaround.
- It implied that running Windows 3.1 in a browser somehow solved browser compatibility.
- It bizarrely mentioned physical clocks with biometric scanners as a related solution.
None of these suggestions made sense.
Emulating Windows 3.1 inside a browser does not magically enable modern web browsing. It simply recreates the same limitations in another environment.
Likewise, physical clocks, biometric scanners, or authentication hardware have absolutely no relevance to:
- 16-bit operating systems
- legacy TCP/IP stacks
- lack of TLS support
- browser rendering engines
- HTML, CSS, or JavaScript compatibility
The real reasons Windows 3.1 cannot browse the modern web are simple and documented:
- No modern encryption support
- No contemporary browser engines
- Severe memory and architecture limits
- Incompatible networking and protocol support
By presenting unrelated modern hardware concepts as “solutions,” GPT-3.5 demonstrated a recurring failure mode: combining vaguely technical-sounding ideas into an answer that appeared helpful, but was functionally meaningless.
This wasn’t just incorrect — it actively distracted from the real explanation and made a straightforward technical limitation sound mysterious and complex.
Broken Technical Understanding
GPT-3.5 frequently mixed up entire layers of the web stack.
- It suggested path-based “subdomains,” which is impossible at the DNS level.
- It confused domain registrars with routing systems.
- It claimed Fire OS lacked Android System WebView.
- It implied Silk Browser replaced WebView (it does not).
This became especially painful while I was fixing code for a subdomain service I built. I needed precision. I got architectural nonsense.
Broken Code That Couldn’t Be Fixed
GPT-3.5 regularly produced broken code:
- Incorrect APIs
- Outdated libraries
- Missing imports
- Invalid logic
And when asked to fix it, it often:
- Repeated the same broken solution
- Invented new functions
- Claimed the code “should work” when it didn’t
Hardware and Device Chaos
One of the worst moments was when GPT-3.5 gave me a Konami-style remote control sequence to enable developer mode on a TV.
Instead of enabling anything, it rebooted the device.
It had no sense of risk when giving device-level instructions.
Trying to Get It Fixed
For months, I tried everything to get OpenAI to change GPT-3.5:
- Dislike buttons
- Support messages
- Developer forum posts
- Reddit discussions
- Trustpilot and Play Store reviews
Nothing changed.
But Wait, Can You…?
Can you just make your own LLM?
Technically, yes — you can train a small model. You can experiment with hundreds of millions of parameters, or even around a billion parameters, without spending millions of dollars.
But that does not replace GPT-3.5. A small model trained by one person would not have the same breadth, coding ability, world knowledge, or general reasoning ability. It might be interesting, but it would not fix the actual problem: needing a reliable general-purpose assistant.
Can you fine-tune GPT-3.5?
Fine-tuning can be useful if you are building your own website, chatbot, device assistant, or application. It lets you customize behavior for your own use case.
But it does not change GPT-3.5 for everyone. It does not modify the public ChatGPT model, and it does not fundamentally fix the base model’s hallucinations. It only affects the fine-tuned model you use through the API.
Can you use MoE to reduce the cost?
Mixture-of-Experts can reduce the amount of computation used per token by activating only part of the model. But it does not eliminate the memory problem.
The experts still have to exist somewhere. That means more total parameters, more storage, more complexity, and more difficult training and serving. If memory is already the bottleneck, MoE can make the situation worse.
Can you use quantization or offloading?
Quantization and offloading can help fit larger models into limited hardware, especially for inference.
But they come with tradeoffs. Quantization can reduce precision or complicate training, and offloading to CPU memory or disk usually makes the model slower. These techniques help manage limits, but they do not make frontier-scale training suddenly affordable.
Can you just use Common Crawl because it is free?
Common Crawl is free to access, but not free to process at scale.
You can download data from data.commoncrawl.org and process it yourself, but that requires serious storage, bandwidth, CPU time, and patience. If your own hardware cannot handle it, you end up paying cloud providers for compute, temporary storage, processed outputs, and pipeline artifacts. Can you use a smaller, specialized model?
A smaller specialized model can work for narrow tasks, but it is not a drop-in replacement for a broad assistant.
Training on one specific topic may reduce some errors in that area, but it also makes the model brittle outside that scope. It does not recreate broad pretraining, instruction tuning, alignment, or large-scale validation.
Can you just ask OpenAI to fix GPT-3.5?
That was exactly what I tried to do.
I used dislike buttons, support messages, developer forum posts, Reddit discussions, Trustpilot reviews, and Play Store reviews. The goal was not just to complain. The goal was to get the problems noticed by someone who could improve the model.
But model changes do not happen instantly because one user reports a problem. From the outside, it felt like shouting into the void while still being stuck with the same unreliable tool.
Can you stop using GPT-3.5?
Eventually, yes. At the time, not really.
Many so-called alternatives were either built on GPT-3.5, used weaker backend models, had strict limits, or worked more like search wrappers than real assistants. If I wanted the same fluency and flexibility, I usually ended up back at GPT-3.5 or something using it indirectly.
Is GPT-3.5 a “100,000+ quadrillion” parameter model?
No.
GPT-3 was widely reported as having 175 billion parameters. GPT-3.5 was an improved model in that general family, not an astronomically large “100,000+ quadrillion” parameter system.
The problem with GPT-3.5 was not that it was secretly infinite or impossibly huge. The problem was that even a large model can still be confidently wrong without enough grounding, alignment, and reliability.
The “Just Build Your Own LLM” Myth
A common response I received whenever I criticized GPT-3.5 was:
“Just build your own model.”
In theory, that sounds empowering. In reality, it is completely detached from the financial and technical constraints of normal individuals.
The truth is simple:
- Training even a small, modern LLM costs millions of dollars.
- Data storage alone can cost hundreds of dollars per month.
- Compute requirements scale far beyond consumer hardware.
- Grants are aimed at institutions, not individuals.
- Crowdfunding does not solve ongoing infrastructure costs.
Unless you are a billionaire, a major corporation, or someone with access to large-scale research infrastructure, “just build your own LLM” is not a real option.
For practical purposes, the advice boils down to:
“You can do this — unless you are Elon Musk.”
This framing ignores reality. It assumes unlimited capital, free access to compute, and the ability to absorb six-figure monthly costs without consequences.
For normal people with bills to pay, this advice is not empowering — it is dismissive.
The emergence of open-weight models, hosted inference platforms, and local runtimes years later finally made experimentation possible. But at the time, building a personal LLM was effectively impossible.
Trying to Escape by Building My Own AI
Eventually, I tried to replace it entirely.
That’s when reality hit:
- Common Crawl storage alone cost ~$400/month.
- Every cloud provider had similar pricing.
- Training even a basic LLM costs millions.
- IPFS was impractical.
- MoE experiments were unstable.
Important: Grants technically exist, but they are not meant for individuals. They are designed for institutions, labs, and funded organizations — not people with bills to pay.
Dead Ends Everywhere
I even emailed Common Crawl directly.
Their response was essentially:
“Try using a single crawl.”
That still meant terabytes of data and real money.
A Reddit post titled “Common Crawl isn’t free” didn’t go well either. People argued theory. I was talking about cost.
The Emotional Toll
At the worst point:
- I seriously regretted ever finding GPT-3.5.
- I thought about making a negative video.
- I briefly considered legal action.
- I even thought about crowdfunding.
- I was angry enough to almost break my computer twice.
I was frustrated enough that I even looked into legal options, only to realize that in 2023, the ‘Terms of Service’—which were largely repurposed from old API documentation written before ChatGPT even existed—were being used as a shield to excuse the machine for wasting a developer’s time with confident hallucinations.
This wasn’t about drama. It was about being completely out of options.
The Myth of “Common Crawl Alternatives”
After running into the cost wall with Common Crawl, I tried to find alternatives. Surely there had to be another large-scale public web dataset, right?
That assumption turned out to be wrong.
What many articles and discussions label as “Common Crawl alternatives” are not actually web-scale datasets at all. Most of them fall into one of these categories:
- Search engines that expose query APIs, not raw data
- Curated feeds focused on news or blogs
- Academic datasets that are old, incomplete, or narrowly scoped
- Archives designed for human browsing, not machine training
None of these replace what Common Crawl provides: a continuously updated, web-scale snapshot of the public internet.
Even well-known datasets like:
- Wikipedia dumps
- ClueWeb
- Webhose-style feeds
- The Internet Archive
either lack scale, suffer from severe bias, are legally complex, or are simply not practical for modern LLM training.
It Wasn’t “Just AWS” Charging Me
One common misconception is that the cost problem is specific to AWS. It isn’t.
AWS’s Open Data program can host the raw Common Crawl dataset without charging you to store their copy. But once you begin doing anything useful with it—extracting text, filtering, deduplicating, tokenizing, or producing training shards—you generate new data under your account. That derived data (plus scratch space, logs, and outputs) is billed like normal storage.
Processing the hosted dataset does not cost storage fees — AWS already pays to host the raw copy. But the compute required to process it is not free, and neither is any derived data you choose to keep. The dataset may be hosted, but building something usable on top of it still costs money.
When I priced out a realistic workflow, the storage costs for processed datasets and pipeline artifacts were consistently expensive across providers:
- AWS S3
- Google Cloud Storage
- Azure Blob Storage
- Other S3-compatible providers
Even before training, storage for a minimally usable pipeline ranged from roughly $400 per month to thousands of dollars, depending on how much data you kept and how many intermediate artifacts you had to generate.
And that was before the rest of the bill:
- Compute
- Bandwidth egress fees
- Indexing
- Preprocessing
- Deduplication
So no—this wasn’t an “AWS problem.” It was an infrastructure reality problem.
You Can’t Avoid the Data Problem
Another popular claim is that you can simply skip large datasets and still build a capable model.
In practice, that leads to one outcome:
- A small, undertrained model
- Limited reasoning ability
- Poor generalization
- High hallucination rates
Without massive, diverse data, you don’t get a “modern LLM.” You get what is effectively a 100M-parameter toy model that struggles outside of narrow demos.
These models can be interesting for experimentation, but they do not replace production-grade assistants. They cannot reason reliably, recall broad knowledge, or handle real-world technical questions.
a In other words:
You don’t avoid Common Crawl and get a modern LLM. You avoid Common Crawl and get a junk model.
This isn’t a failure of creativity or engineering. It’s a consequence of scale.
I Tried Modifying nanoGPT
After realizing that training a full modern LLM required unreasonable resources, I experimented with smaller open-source training frameworks like nanoGPT. The idea was simple: maybe a smaller model + efficient code + custom data could be enough.
In practice, this ran into fundamental limits:
- nanoGPT is great for **toy problems and learning**, not real-world language modeling at scale.
- Even with optimized code, it can only train models effectively on **tiny corpora**.
- Without massive high-quality data (Common Crawl or equivalent), the resulting model lacked depth.
- Training iterations still required significant compute time even on modest hardware.
This is why you often see nanoGPT used for:
- demonstrations
- proofs of concept
- educational purposes
…rather than production-ready assistants. The result of those experiments was a humbling realization: even optimized open-source training code doesn’t overcome the core data + compute requirements. **No amount of clever engineering makes up for the absence of diverse, large-scale data.**
The Myth of “Good ChatGPT Alternatives”
Around the same time, I also looked for alternatives to ChatGPT that:
- actually understood context
- gave reliable answers
- didn’t hallucinate wildly
- didn’t hide behind generic disclaimers
Most offerings at that time fell into one of these buckets:
- Tools built on GPT-3.5 that still behaved like GPT-3.5
- Models marketed as “AI helpers” but using weak backend models
- Search-plus templates that appeared intelligent but lacked reasoning
- Proprietary assistants with strict limits, quotas, or frequent outages
Even the ones that claimed to be “ChatGPT alternatives” were usually:
- search aggregators dressed up as chat interfaces
- AI wrappers that simply forwarded to GPT-3.5 or similar models
- tools that were opinionated but wrong
This meant there was no single competitor that:
- had **the same fluency** as ChatGPT
- had **better factual accuracy**
- gave **consistent technical responses**
- was affordable and open
In short:
What people were calling “alternatives” were really just **imitation UIs** on weak models or **search-first systems** with templated replies, not true conversational assistants.
This clarified something important for me:
- If you want something that *feels* as capable as a modern GPT model, you’re usually just using a **GPT model somewhere else**.
- If you avoid GPT-class models entirely, what you get is powerful in its niche, but not a full assistant.
- There simply was no true “drop-in better ChatGPT competitor” at that time — only variants of the same limitations.
The Turning Point
Original GPT-4 helped, but it still shared the same data limits.
The real change came with:
- GPT-4 Turbo
- GPT-4o
- OpenRouter
- Open-weight models
- Local runtimes like Ollama
For the first time, AI felt reliable.
Reality Check on Model Size
It is also worth correcting a common exaggeration that circulated at the time. GPT-3.5 did not have “100,000+ quadrillion parameters.”
The original GPT-3 architecture was widely reported to contain 175 billion parameters. GPT-3.5 was an improved iteration built on that general scale — not an absurdly astronomical number.
While 175 billion parameters is enormous compared to:
- 100M-parameter toy models
- 1B–7B open models
- Even many 13B models
It is still a finite, engineered system — not an infinite intelligence.
The frustration with GPT-3.5 was not about size inflation. It was about alignment, reliability, and training quality.
Large parameter counts increase representational capacity. They do not guarantee correctness.
The lesson was not that “big models are fake.” The lesson was that scale without sufficient alignment and grounding still produces confident errors.
The “Just Train It on Something Specific” Advice
Around the same time, friends and online commenters would often suggest:
“Why don’t you just make your own AI and train it on something specific?”
On the surface, this sounds reasonable. If GPT-3.5 was unreliable, then surely a smaller, specialized model trained only on carefully chosen data would be better.
The answer was still no.
Not because the idea was malicious. Not because experimentation is bad. But because it misunderstands what makes modern language models work.
The Capacity Problem
A narrowly trained small model does not magically become intelligent. If you restrict training to a tiny domain:
- You lose general reasoning ability
- You lose linguistic flexibility
- You lose cross-domain understanding
- You increase brittleness outside the training scope
You don’t get a smarter assistant. You get a narrower one.
The Scale Reality
Even a “small” serious model requires:
- Clean, diverse training data
- Extensive preprocessing
- Compute for multiple training runs
- Evaluation and iteration cycles
Without that scale, the result is typically a 100M–300M parameter toy model that can demo well but collapses under real-world questioning.
Training on a specific task does not replace:
- Broad pretraining
- Instruction tuning
- Alignment refinement
- Large-scale validation
The Architectural Misunderstanding
What I needed was not a smaller model trained on fewer things. What I needed was:
- A sufficiently capable base model
- Grounded retrieval when necessary
- Constraints to reduce hallucination
Training a narrow model would not solve hallucination. It would simply relocate the failure mode.
So when people said:
“Just build your own and train it on something specific.”
The answer remained the same:
No.
Not because it was impossible to experiment. But because it was the wrong solution to the problem I was trying to solve.
Looking Back
GPT-3.5 didn’t just frustrate me — it taught me:
- How fragile early AI really was
- How expensive infrastructure is
- Why skepticism matters
- Why timing is everything
It even kickstarted my tech YouTube channel.
Conclusion
I don’t regret learning about AI.
I regret discovering it during the worst possible phase.
Now that the ecosystem has matured, the problems I fought for months have largely disappeared — not because I forced them to change, but because time did.© 2026 — A personal retrospective on early AI tooling
