US Patent Issued to Microsoft Technology Licensing on Nov. 3 for "Systems and methods for managing multi-tier key-value caching for generative AI models" (Massachusetts, California, Florida Inventors)

ALEXANDRIA, Va., Nov. 3 -- United States Patent no. 12,783,414, issued on Nov. 3, was assigned to Microsoft Technology Licensing LLC (Redmond, Wash.).

"Systems and methods for managing multi-tier key-value caching for generative AI models" was invented by Hokchhay Tann (North Reading, Mass.), Hima Bindu Damecharla (San Diego), Preyas Janak Shah (Odessa, Fla.) and Matthew Mattina (Boylston, Mass.).

According to the abstract* released by the U.S. Patent & Trademark Office: "Systems and methods for managing multi-tier key-value (KV) caching for a generative artificial intelligence (AI) model are described. A system includes a lower tier memory, a higher tier memory, and a KV cache manager to: (1) determine whether a key-value (KV) cache corr...