Other Software Projects Are Now Trying to Replicate ChatGPT
Published on January 01, 2023 at 01:04AM
"The first open source equivalent of OpenAI's ChatGPT has arrived," writes TechCrunch, "but good luck running it on your laptop — or at all." This week, Philip Wang, the developer responsible for reverse-engineering closed-sourced AI systems including Meta's Make-A-Video, released PaLM + RLHF, a text-generating model that behaves similarly to ChatGPT [listed as a work in progress]. The system combines PaLM, a large language model from Google, and a technique called Reinforcement Learning with Human Feedback — RLHF, for short — to create a system that can accomplish pretty much any task that ChatGPT can, including drafting emails and suggesting computer code. But PaLM + RLHF isn't pre-trained. That is to say, the system hasn't been trained on the example data from the web necessary for it to actually work. Downloading PaLM + RLHF won't magically install a ChatGPT-like experience — that would require compiling gigabytes of text from which the model can learn and finding hardware beefy enough to handle the training workload.... PaLM + RLHF isn't going to replace ChatGPT today — unless a well-funded venture (or person) goes to the trouble of training and making it available publicly. In better news, several other efforts to replicate ChatGPT are progressing at a fast clip, including one led by a research group called CarperAI. In partnership with the open AI research organization EleutherAI and startups Scale AI and Hugging Face, CarperAI plans to release the first ready-to-run, ChatGPT-like AI model trained with human feedback. LAION, the nonprofit that supplied the initial dataset used to train Stable Diffusion, is also spearheading a project to replicate ChatGPT using the newest machine learning techniques.
Published on January 01, 2023 at 01:04AM
"The first open source equivalent of OpenAI's ChatGPT has arrived," writes TechCrunch, "but good luck running it on your laptop — or at all." This week, Philip Wang, the developer responsible for reverse-engineering closed-sourced AI systems including Meta's Make-A-Video, released PaLM + RLHF, a text-generating model that behaves similarly to ChatGPT [listed as a work in progress]. The system combines PaLM, a large language model from Google, and a technique called Reinforcement Learning with Human Feedback — RLHF, for short — to create a system that can accomplish pretty much any task that ChatGPT can, including drafting emails and suggesting computer code. But PaLM + RLHF isn't pre-trained. That is to say, the system hasn't been trained on the example data from the web necessary for it to actually work. Downloading PaLM + RLHF won't magically install a ChatGPT-like experience — that would require compiling gigabytes of text from which the model can learn and finding hardware beefy enough to handle the training workload.... PaLM + RLHF isn't going to replace ChatGPT today — unless a well-funded venture (or person) goes to the trouble of training and making it available publicly. In better news, several other efforts to replicate ChatGPT are progressing at a fast clip, including one led by a research group called CarperAI. In partnership with the open AI research organization EleutherAI and startups Scale AI and Hugging Face, CarperAI plans to release the first ready-to-run, ChatGPT-like AI model trained with human feedback. LAION, the nonprofit that supplied the initial dataset used to train Stable Diffusion, is also spearheading a project to replicate ChatGPT using the newest machine learning techniques.
Read more of this story at Slashdot.
Comments
Post a Comment