US Patent Issued to Databricks on June 2 for "Updating KV-cache across transformer layers" (California Inventors)

ALEXANDRIA, Va., June 2 -- United States Patent no. 12,645,592, issued on June 2, was assigned to Databricks Inc. (San Francisco).

"Updating KV-cache across transformer layers" was invented by Vitaliy A. Chiley (San Diego) and Shashank Rajput (San Francisco).

According to the abstract* released by the U.S. Patent & Trademark Office: "A data processing service generates keys and values for decoder layers. The service receives a request from a client device. The service accesses a ML transformer architecture which includes decoder layers. Each decoder layer is associated with a KV-cache. The service applies the transformer architecture to input tokens obtained from the request to generate output tokens. For each decoder layer, the service g...