Parallel and Distributed Deep Learning Paper Database
The paper database below was collected for the purpose of the paper "Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis".
It contains works that utilize parallel and distributed computing resources for training Deep Neural Networks. This includes hardware architectures,
data representation, parallelization strategies, distributed algorithms, system implementations, frameworks, and programming models.
Format
The papers are organized in the YAML format, with one entry per paper, sorted by publication year. Each entry contains the paper title, year of publication, category (corresponding to sections in the paper), keywords, datasets used in experiments, frameworks, and hardware architectures.
The following listing shows an example of such a paper entry:
'Large Scale Distributed Deep Networks':
year: 2012
categories:
- Distributed
- Systems
keywords:
- Asynchronous SGD
- DistBelief
- Downpour SGD
- Sandblaster LBFGS
- Parameter server
- Model-parallelism
- Data-parallelism
- Layer pipelining
- Hybrid parallelism
hardware:
- CPU Cluster:
nodes: 5100
commlayer: Sockets
experiments:
datasets:
- Speech Recognition (internal)
- ImageNet
networks:
- 4-layer MLP
- LCN
Download
References
