OpenAI Blog · Dec 6, 2017
Block-sparse GPU kernels
Reviewed by Errol Vogt, Site support technician & online learning analyst · original summary · editorial policy
Block-sparse GPU kernels. We’re releasing highly-optimized GPU kernels for an underexplored class of neural network architectures: networks with block-sparse weights. Depending on the chosen sparsity, these kernels can run orders of magnitude faster than cuBLAS or cuSPARSE. We’ve used them to attain state-of-the-art results in text sentiment analysis and generative modeling of text and images. This update is relevant for small-office operators tracking changes in their tools.
Operator takeaway: For operators: review whether 'Block-sparse GPU kernels' affects your current setup before relying on it in production.
ai