iot.eetimes.com iot.eetimes.com

Training a 20–Billion Parameter AI Model on a Single Processor

This article originally appeared on EETimes. Cerebras has shown off the capabilities of its second–generation wafer–scale engine, announcing it has set the record for the largest AI model ever trained on a single device. For the first time, a natural language processing network with 20 billion parameters, GPT–NeoX 20B, was trained on a single device. Here’s why that matters. WHY DO WE NEED TO TRAIN MODELS THIS BIG? A new type...