US Patent Issued to Amazon Technologies on Nov. 3 for "Speculative decoding techniques for generative models" (Washington, New Jersey, Minnesota, Hawaii, Massachusetts, California Inventors)

ALEXANDRIA, Va., Nov. 3 -- United States Patent no. 12,783,803, issued on Nov. 3, was assigned to Amazon Technologies Inc. (Seattle).

"Speculative decoding techniques for generative models" was invented by Saket Dingliwal (Seattle), Sravan Babu Bodapati (Fremont, Calif.), Shubham Katiyar (Bothell, Wash.), Spurthideepika Sandiri (Holliston, Mass.), Venkata Siva Sai Krishna Balakavi (Jersey City, N.J.), Melanie C B Gens (Honolulu) and Samuel Tucker (Minneapolis).

According to the abstract* released by the U.S. Patent & Trademark Office: "Techniques for improving speculative decoding for improved compute utilization are described. A system may determine a compute budget representing a maximum number of speculative tokens to be verified based...