Intelligent Mutations in Genetic Programming: OpenAI Proposes Evolution Through Large Models
Large-scale language models (LLMs) have achieved impressive performance in automated code generation by bootstrapping human knowledge and learning from extremely large datasets. Might it be possible to combine deep...
A WaveNet Rival? Stanford U Study Models Raw Audio Waveforms Over Contexts of 500k Samples
The effective modelling of long-term dependencies enables conditioning new model outputs on previous inputs and is critical when dealing with longer text, audio or video contexts. However, when modelling the long-term...
DeepMind Boosts RL Agents’ Retrieval Capability to Tens of Millions of Pieces of Information
The conventional approach for improving the decision-making of deep reinforcement learning (RL) agents is to gradually amortize the useful information they gain from their experiences via gradient descent on training...
Meta AI’s LegoNN Builds Decoder Modules That Are Reusable Across Diverse Language Tasks Without Fine-Tuning
Encoder-decoder models have become the preferred approach for a wide range of language-related tasks. Although some common logical functions are shared between different tasks, most contemporary encoder-decoder models...
Google Leverages Transformers to Vastly Simplify Neural Video Compression With SOTA Results
Neural network-based approaches have made significant progress on video compression over the last several years, reaching performance on par with classical codec-based methods. These novel neural approaches however are...
Wav2Vec 2.0 Learns Brain-Like Representations From Just 600 Hours of Unlabeled Speech Data in New Study
Deep neural networks have recently hinted at their potential for processing speech in a manner more like the human brain and generating activations similar to those of the brain in response to the same inputs. The...
Apple’s MobileOne Backbone Reduces Inference Time to Under One Millisecond on an iPhone12 and Reaches 75.9% Top-1 Accuracy on ImageNet
As AI systems increasingly move from the cloud to devices, identifying suitable neural network backbones for mobile device deployment has become a hot research area. While decreasing floating-point operations (FLOPs)...
444 Authors From 132 Institutions Release BIG-bench: A 204-Task ‘Extremely Difficult and Diverse’ Benchmark for Large Language Models
Powered by their ever-increasing scale, today’s large language models have shown breakthrough capabilities beyond natural language processing (NLP), in areas such as writing computer code, diagnosing medical...