サクサク読めて、アプリ限定の機能も多数!
トップへ戻る
GPT-4o
distill.pub
This article is one of two Distill publications about graph neural networks. Take a look at A Gentle Introduction to Graph Neural Networks for a companion view on many things graph and neural network related. Many systems and interactions - social networks, molecules, organizations, citations, physical models, transactions - can be represented quite naturally as graphs. How can we reason about and
Neural networks have been adapted to leverage the structure and properties of graphs. We explore the components needed for building a graph neural network - and motivate the design choices behind them. Hover over a node in the diagram below to see how it accumulates information from nodes around it through the layers of the network. Authors Affiliations Benjamin Sanchez-Lengeling Google Research E
This article is part of the Differentiable Self-organizing Systems Thread, an experimental format collecting invited short articles delving into differentiable self-organizing systems, interspersed with critical commentary from several experts in adjacent fields. Self-classifying MNIST Digits Adversarial Reprogramming of Neural Cellular Automata Neural Cellular Automata (NCA We use NCA to refer to
Examining the design of interactive articles by synthesizing theory from disciplines such as education, journalism, and visualization. Computing has changed how people communicate. The transmission of news, messages, and ideas is instant. Anyone’s voice can be heard. In fact, access to digital communication technologies such as the Internet is so fundamental to daily life that their disruption by
Breaking Bayesian Optimization into small, sizeable chunks. Authors Affiliations Apoorv Agnihotri Indian Insitute of Technology Gandhinagar Nipun Batra Indian Insitute of Technology Gandhinagar Many modern machine learning algorithms have a large number of hyperparameters. To effectively use these algorithms, we need to pick good hyperparameter values. In this article, we talk about Bayesian Optim
Growing models were trained to generate patterns, but don't know how to persist them. Some patterns explode, some decay, but some happen to be almost stable or even regenerate parts! [experiment 1] Persistent models are trained to make the pattern stay for a prolonged period of time. Interstingly, they often develop some regenerative capabilities without being explicitly instructed to do so [exper
Introduction In the last few years, reinforcement learning (RL) has made remarkable progress, including beating world-champion Go players, controlling robotic hands, and even painting pictures. One of the key sub-problems of RL is value estimation – learning the long-term consequences of being in a state. This can be tricky because future returns are generally noisy, affected by many things other
What we’d like to find out about GANs that we don’t know yet. Problem 1What are the trade-offs between GANs and other generative models? Problem 2What sorts of distributions can GANs model? Problem 3How can we Scale GANs beyond image synthesis? Problem 4What can we say about the global convergence of the training dynamics? Problem 5How should we evaluate GANs and when should we use them? Problem 6
How to turn a collection of small building blocks into a versatile tool for solving regression problems. Even if you have spent some time reading about machine learning, chances are that you have never heard of Gaussian processes. And if you have, rehearsing the basics is always a good way to refresh your memory. With this blog post we want to give an introduction to Gaussian processes and make th
Inspecting gradient magnitudes in context can be a powerful tool to see when recurrent units use short-term or long-term contextual understanding. This connectivity visualization shows how strongly previous input characters influence the current target character in an autocomplete problem. For example, in the prediction of “grammar” the GRU RNN initially uses long-term memorization but as more cha
Exploring Neural Networks with Activation Atlases By using feature inversion to visualize millions of activations from an image classification network, we create an explorable activation atlas of features the network has learned which can reveal how the network typically represents some concepts. Above, an activation atlas of the InceptionV1 vision classification network reveals many fully realize
A powerful, under-explored tool for neural network visualizations and art. Neural networks trained to classify images have a remarkable — and surprising! — capacity to generate images. Techniques such as DeepDream , style transfer, and feature visualization leverage this capacity as a powerful tool for exploring the inner workings of neural networks, and to fuel a small artistic movement based on
Interpretability techniques are normally studied in isolation. We explore the powerful interfaces that arise when you combine them — and the rich structure of this combinatorial space. With the growing success of neural networks, there is a corresponding need to be able to explain their decisions — including building confidence about how they will behave in the real-world, detecting model bias, an
Sentence from Pride and Prejudice by Jane Austen. Interpolated by the authors. Inspired by experiments done by the novelist Robin Sloan Such generative interfaces provide a kind of cartography of generative models, ways for humans to explore and make meaning using those models. We saw earlier that the font model automatically infers relatively deep principles about font design, and makes them avai
Introduction Consider speech recognition. We have a dataset of audio clips and corresponding transcripts. Unfortunately, we don’t know how the characters in the transcript align to the audio. This makes training a speech recognizer harder than it might at first seem. Without this alignment, the simple approaches aren’t available to us. We could devise a rule like “one character corresponds to ten
If we want to understand individual features, we can search for examples where they have high values — either for a neuron at an individual position, or for an entire channel. We used the channel objective to create most of the images in this article. If we want to understand a layer as a whole, we can use the DeepDream objective , searching for images the layer finds “interesting.” And if we want
We often think of Momentum as a means of dampening oscillations and speeding up the iterations, leading to faster convergence. But it has other interesting behavior. It allows a larger range of step-sizes to be used, and creates its own oscillations. What is going on? Here’s a popular story about momentum [1, 2, 3]: gradient descent is a man walking down a hill. He follows the steepest path downwa
Machine Learning Research Should Be Clear, Dynamic and Vivid. Distill Is Here to Help.
Neural networks are an extremely successful approach to machine learning, but it’s tricky to understand why they behave the way they do. This has sparked a lot of interest and effort around trying to understand and visualize them, which we think is so far just scratching the surface of what is possible. In this article we will try to push forward in this direction by taking a generative model of h
Understanding Convolutions on Graphs Ameya Daigavane, Balaraman Ravindran, and Gaurav Aggarwal Understanding the building blocks and design choices of graph neural networks. A Gentle Introduction to Graph Neural Networks Benjamin Sanchez-Lengeling, Emily Reif, Adam Pearce, and Alexander B. Wiltschko What components are needed for building learning algorithms that leverage the structure and propert
When we look very closely at images generated by neural networks, we often see a strange checkerboard pattern of artifacts. It’s more obvious in some cases than others, but a large fraction of recent models exhibit this behavior. Mysteriously, the checkerboard pattern tends to be most prominent in images with strong colors. What’s going on? Do neural networks hate bright colors? The actual cause o
A popular method for exploring high-dimensional data is something called t-SNE, introduced by van der Maaten and Hinton in 2008 [1]. The technique has become widespread in the field of machine learning, since it has an almost magical ability to create compelling two-dimensonal “maps” from data with hundreds or even thousands of dimensions. Although impressive, these images can be tempting to misre
Recurrent neural networks are one of the staples of deep learning, allowing neural networks to work with sequences of data like text, audio and video. They can be used to boil a sequence down into a high-level understanding, to annotate sequences, and even to generate new sequences from scratch! x0 y0 x1 y1 x2 y2 x3 y3 One cell... can be used over... and over... and over... x4 y4 again. The basic
このページを最初にブックマークしてみませんか?
『Distill — Latest articles about machine learning』の新着エントリーを見る
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く