Gradients without Backpropagation
Using backpropagation to compute gradients of objective functions for optimization has remained a mainstay of machine learning. Backpropagation, or reverse-mode differentiation, is a special case within the general family of automatic differentiation algorithms that also includes the forward mode. We present a method to compute gradients based solely on the directional derivative that one can comp
Introducing TensorFlow Graph Neural Networks
Posted by Sibon Li, Jan Pfeifer and Bryan Perozzi and Douglas Yarrington Today, we are excited to release TensorFlow Graph Neural Networks (GNNs), a library designed to make it easy to work with graph structured data using TensorFlow. We have used an earlier version of this library in production at Google in a variety of contexts (for example, spam and anomaly detection, traffic estimation, YouTub
RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients?
Recurrent neural networks (RNNs) are particularly well-suited for modeling long-term dependencies in sequential data, but are notoriously hard to train because the error backpropagated in time either vanishes or explodes at an exponential rate. While a number of works attempt to mitigate this effect through gated recurrent units, well-chosen parametric constraints, and skip-connections, we develop
Anchor Boxes — The key to quality object detection
One of the hardest concepts to grasp when learning about Convolutional Neural Networks for object detection is the idea of anchor boxes. It is also one of the most important parameters you can tune for improved performance on your dataset. In fact, if anchor boxes are not tuned correctly, your neural network will never even know that certain small, large or irregular objects exist and will never h
要注目？DeepGBM: ニューラルネット+GBDT（速報） - Paoの技術力を磨くブログ
KDD2019のPaper一覧で気になるものがあったので紹介します。 ※記載時点でまだ論文公開、発表されておらず、こちら鮮度重視の記事です。 内容に誤りがある可能性は十分あるのでご了承ください。 DeepGBMとは データマイニングのトップカンファレンスKDD2019で発表される予定の手法です。 Guolin Ke, Zhenhui Xu, Jia Zhang, Jiang Bian, and Tie-yan Liu. "DeepGBM: A Deep Learning Framework Distilled by GBDT for Online Prediction Tasks." In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM
AutoML | Literature on Neural Architecture Search
Maintained by Difan Deng and Marius Lindauer. The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know. Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization.
Visualizing memorization in RNNs
Inspecting gradient magnitudes in context can be a powerful tool to see when recurrent units use short-term or long-term contextual understanding. This connectivity visualization shows how strongly previous input characters influence the current target character in an autocomplete problem. For example, in the prediction of “grammar” the GRU RNN initially uses long-term memorization but as more cha
Chatbot and Related Research Paper Notes with Images
Papers related to chatbot models in chronological order spanning about 5 years from 2014. Some papers are not about chatbots, but I included them because they are interesting, and they may provide insights into creating new and different conversation models. For each paper I provided a link, the names of the authors, and GitHub implementations of the paper (noting the deep learning framework) if I
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
The rising popularity of intelligent mobile devices and the daunting computational cost of deep learning-based models call for efficient and accurate on-device inference schemes. We propose a quantization scheme that allows inference to be carried out using integer-only arithmetic, which can be implemented more efficiently than floating point inference on commonly available integer-only hardware.
統計的声質変換の紹介 - nico-opendata
nico-opendata niconicoでは、学術分野における技術発展への寄与を目的として、 研究者の方を対象に各種サービスのデータを公開しています。 ニコニコ動画コメント等データセット (株)ドワンゴ及び(有)未来検索ブラジルと国立情報学研究所が協力して研究者に提供しているデータセットです。 ニコニコ動画コメント等のデータが利用可能です。 利用申請フォーム※国立情報学研究所へリンクします ニコニコ大百科データ (株)ドワンゴ及び(有)未来検索ブラジルと国立情報学研究所が協力して研究者に提供しているデータセットです。 ニコニコ大百科のデータが利用可能です。 利用申請フォーム※国立情報学研究所へリンクします Nico-Illustデータセット Comicolorization: Semi-Automatic Manga Colorization Chie Furusawa*、Kazuyu
DNN推論用ライブラリ「Menoh」リリースについて - Preferred Networks Research & Development
Python以外も使いたくないですか？ 特にDeepLearning界隈で． Menoh開発者の岡田です．この記事ではMenohの紹介と開発に至った動機について説明します． Menohのレポジトリ： https://github.com/pfnet-research/menoh Menoh（メノウ）は学習済みのDNNモデルをONNX形式から読み込んで動作させる推論専用のライブラリです．実装はC++で書きましたが，C言語のインターフェースを持たせて，他の言語用からもその機能を呼び出しやすくしてあります．リリース時点でC++版ラッパーとC#版ラッパー，Haskell版ラッパーがあり，Ruby版ラッパーとNodeJS版ラッパー，Java（JVM）版ラッパーが開発中です．バックエンドにはIntelの開発しているMKL-DNNを採用し，GPUが無くてもIntel CPUが使える環境で高速にモデルの
Hyperbolic Attention Networks
We introduce hyperbolic attention networks to endow neural networks with enough capacity to match the complexity of data with hierarchical and power-law structure. A few recent approaches have successfully demonstrated the benefits of imposing hyperbolic geometry on the parameters of shallow networks. We extend this line of work by imposing hyperbolic geometry on the activations of neural networks
WaveGAN Demo
Please try again on a different browser or refresh if you change your mind :) Chris Donahue, Julian McAuley, Miller Puckette This is a demo of our WaveGAN method trained on drum sound effects (paper, code). All drum sounds are synthesized in browser by a neural network. Shortcuts: Keys 1-8 play sounds. Shift+[1-8] changes sounds. Space starts/stops sequencer.
DensePose: Dense Human Pose Estimation In The Wild
In this work, we establish dense correspondences between RGB image and a surface-based representation of the human body, a task we refer to as dense human pose estimation. We first gather dense correspondences for 50K persons appearing in the COCO dataset by introducing an efficient annotation pipeline. We then use our dataset to train CNN-based systems that deliver dense correspondence 'in the wi
DensePose
Dense human pose estimation aims at mapping all human pixels of an RGB image to the 3D surface of the human body. We introduce DensePose-COCO, a large-scale ground-truth dataset with image-to-surface correspondences manually annotated on 50K COCO images. We propose DensePose-RCNN, a variant of Mask-RCNN, to densely regress part-specific UV coordinates within every human region at multiple frames p
Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes
Recurrent neural networks are nowadays successfully used in an abundance of applications, going from text, speech and image processing to recommender systems. Backpropagation through time is the algorithm that is commonly used to train these networks on specific tasks. Many deep learning frameworks have their own implementation of training and sampling procedures for recurrent neural networks, whi
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
最近のDeep Learning (NLP) 界隈におけるAttention事情
Graph Neural Networks概観
[DL Hacks]BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding![[DL Hacks]BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://cdn-ak-scissors.b.st-hatena.com/image/square/b6daf77a3db668161a6707d95bec4f65d9f8bdcf/height=288;version=1;width=512/https%3A%2F%2Fcdn.slidesharecdn.com%2Fss_thumbnails%2F20181129suzukibert1-181204064830-thumbnail.jpg%3Fwidth%3D640%26height%3D640%26fit%3Dbounds)
http://archive.xsig.ipsj.or.jp/history/ACSI/2016/download/ACSI2016-tutorial2.pdf