Numenta’s AI technology also dramatically accelerates NLP applications that rely on analyzing extensive collections of documents.1 For example, when applying transformers to document understanding, long sequence lengths are required to incorporate the full context of the document. These long sequences require high data transfer rates, and off-chip bandwidth thus becomes the limiting factor. Using the new Intel® Xeon® CPU Max Series, Numenta demonstrates it can optimize the BERT-Large model to process large text documents, enabling unparalleled 20x throughput speedup for long sequence lengths o
Numenta Delivers Powerful Inference Performance… This synergistic combination of algorithm and hardware advances led to unparalleled performance gains for BERT-Large inference on CPUs. With this dramatic performance acceleration, CPUs become not only a viable alternative to GPUs, but an ideal option for running transformers in production. …
… Part of this is custom-training the model with hardware-aware optimized algorithms. And part of it entails optimizing runtime to fully utilize the computing platform for inference. Their brain-based techniques applied to a custom-trained version of BERT-Large enabled higher throughput in production. …
… This built-in AI accelerator supports the quantization of models to the brain floating 16 bfloat16 or bf16 numeric data type, enhancing the speed and efficiency of AI workloads. …