US Patent Issued to The Toronto-Dominion Bank on June 3 for "Initialization of parameters for machine-learned transformer neural network architectures" (Canadian Inventors)

ALEXANDRIA, Va., June 4 -- United States Patent no. 12,321,861, issued on June 3, was assigned to The Toronto-Dominion Bank (Toronto).

"Initialization of parameters for machine-learned transformer neural network architectures" was invented by Maksims Volkovs (Toronto), Xiao Shi Huang (Toronto) and Juan Felipe Perez Vallejo (Toronto).

According to the abstract* released by the U.S. Patent & Trademark Office: "An online system trains a transformer architecture by an initialization method which allows the transformer architecture to be trained without normalization layers of learning rate warmup, resulting in significant improvements in computational efficiency for transformer architectures. Specifically, an attention block included in an en...