US Patent No. 11,030,529

EVOLUTION OF ARCHITECTURES FOR MULTITASK NEURAL NETWORKS


Patent No. 11,030,529
Issue Date June 08, 2021
Title Evolution Of Architectures For Multitask Neural Networks
Inventorship Jason Zhi Liang, Fremont, CA (US)
Elliot Meyerson, San Francisco, CA (US)
Risto Miikkulainen, Stanford, CA (US)
Assignee Cognizant Technology Solutions U.S. Corporation, College Station, TX (US)

Claim of US Patent No. 11,030,529

1. A processor implemented method for evolving task-specific topologies in a multitask architecture comprising:establishing a set of shared modules which are shared among each task-specific topology;
initializing the shared modules {k}k=1K with random weights;
initializing a champion individual module routing scheme for each task (t), wherein the ith individual for the tth task is represented by a tuple (Eti, Gti,Dti), and further wherein Eti is an encoder, Gti is a DAG, which specifies the individual module routing scheme, and Dti is a decoder, with Eti and Dti initialized with random weights;
for each champion individual (Eti, Gti, Dti), generating a challenger (Et2, Gt2, Dt2) by mutating the tth champion in accordance with a predetermined mutation subprocess;
jointly training each champion and challenger for M iterations on a training set of data;
evaluating each champion and challenger on a validation set of data to determine an accuracy fitness for each individual champion and challenger for its predetermined task;
if a challenger has higher accuracy fitness than a corresponding champion, then the champion is replaced wherein (Eti, Gti, Dti)=(Et2, Gt2, Dt2);
calculating an average accuracy fitness across all champions for tasks in the multitask architecture; and
checkpointing the shared modules when the average accuracy is best achieved.