US Patent Issued to GOOGLE on Aug. 11 for "Full attention with sparse computation cost" (Canadian, American Inventors)

ALEXANDRIA, Va., Aug. 12 -- United States Patent no. 12,705,421, issued on Aug. 11, was assigned to GOOGLE LLC (Mountain View, Calif.).

"Full attention with sparse computation cost" was invented by Hanjun Dai (San Jose, Calif.), Bo Dai (San Jose, Calif.), Hongyu Ren (Stanford, Calif.), Dale Eric Schuurmans (Edmonton, Canada), Zihang Dai (Cupertino, Calif.) and Mengjiao Yang (Berkeley, Calif.).

According to the abstract* released by the U.S. Patent & Trademark Office: "The present disclosure is directed to machine learning model architectures which provide full attention capability in each attention head while maintaining low computation and memory complexity. Specifically, according to one aspect of the present disclosure, example attenti...