US Patent Issued to BEIJING TRANSTREAMS TECHNOLOGY Co. LTD. search on June 23 for "Methods and devices for accelerating a transformer with a sparse attention pattern" (Texas, California Inventors)

ALEXANDRIA, Va., July 15 -- United States Patent no. 12,664,410, issued on June 23, was assigned to BEIJING TRANSTREAMS TECHNOLOGY Co. LTD. search (Beijing).

"Methods and devices for accelerating a transformer with a sparse attention pattern" was invented by Zhendong Wang (Plano, Texas), Yongxiong Ren (San Jose, Calif.), Yang Liu (San Jose, Calif.) and Lingzhi Liu (San Jose, Calif.).

According to the abstract* released by the U.S. Patent & Trademark Office: "A method and an apparatus for accelerating a transformer with a sparse attention pattern are provided. The method includes that a heterogeneous device including one or more GPUs loads a first matrix, a second matrix, and a transformed sparsity mask into a first sampled dense-dense mat...