mastodon.ie is one of the many independent Mastodon servers you can use to participate in the fediverse.
Irish Mastodon - run from Ireland, we welcome all who respect the community rules and members.

Administered by:

Server stats:

1.5K
active users

#gpt2

0 posts0 participants0 posts today
Continued thread

The next chapter in Karpathy's tutorial explains how to reproduce a model closely resembling #OpenAI's original #GPT2.

...but I'm *NOT* trying this on a desktop with a single GPU. The README informs us that this training takes about 4 days on a beefy node with 8 x A100 40GB. Nope!

github.com/karpathy/nanoGPT?ta
#AI #LLM #GPT

GitHubGitHub - karpathy/nanoGPT: The simplest, fastest repository for training/finetuning medium-sized GPTs.The simplest, fastest repository for training/finetuning medium-sized GPTs. - karpathy/nanoGPT

How to break an AI (the illustrated guide 🤖 )

I am posting this for fun, to show how fragile #AI systems are, and how ridiculous it is to imply that they are intelligent or could wipe us out.

1) Grab a model. For this demo, I will take GPT2 because it fits on my laptop.

2) Copy-paste code for running and fine-tuning the AI. You can take mine here, which will also download #GPT2 for you: github.com/possible-worlds-res.

Optional: see how the model, for now, is working as it should… 1/4

GitHubGitHub - possible-worlds-research/AI-buster: Some scripts to demonstrate the instability of Transformers / LLMs.Some scripts to demonstrate the instability of Transformers / LLMs. - possible-worlds-research/AI-buster

If you think #ChatGPT based on #GPT3 is so powerful, think about its predecessor, #GPT2, which was "too dangerous to release" back in 2019... independent.co.uk/tech/ai-arti

Fun fact: the article's URL contains `elon-musk`. No mention of Musk in the article whatsoever...

The IndependentAI deemed ‘too dangerous to release’ makes it out into the worldBy Andrew Griffin

#ICYMI

In case you want to experiment with (so-called) #AI models, a lot of them have been released by huggingface.co/

all available directly via #python packages

for example #gpt2 for machine-generated absurdity (#dada) huggingface.co/gpt2

No #GPU required, works just fine on a normal CPU

huggingface.coHugging Face – The AI community building the future.We’re on a journey to advance and democratize artificial intelligence through open source and open science.

I was able to get #OpenAI’s #GPT2 running locally yesterday. Getting coherent results from #ChatGPT and #GPT3 is amazing, but half the fun with AI writing back in the day was how high the writing-style was.
GPT3 just gets boring after a while, especially when you consider the fact you can’t run it locally and some content gets completely censored. Something feels amazing about being able to just do it on your own laptop without any cloud involved.