US Patent No. 10,366,084

OPTIMIZING PIPELINING RESULT SETS WITH FAULT TOLERANCE IN DISTRIBUTED QUERY EXECUTION


Patent No. 10,366,084
Issue Date July 30, 2019
Title Optimizing Pipelining Result Sets With Fault Tolerance In Distributed Query Execution
Inventorship Zhicheng Yin, Kirkland, WA (US)
Jin Sun, Sammamish, WA (US)
Ming Li, Issaquah, WA (US)
Jaliya Nishantha Ekanayake, Redmond, WA (US)
Yongchul Kwon, Kirkland, WA (US)
Marc T. Friedman, Seattle, WA (US)
Haibo Lin, Issaquah, WA (US)
Raghunath Ramakrishnan, Bellevue, WA (US)
Tao Guan, Renton, WA (US)
Assignee MICROSOFT TECHNOLOGY LICENSING, LLC, Redmond, WA (US)

Claim of US Patent No. 10,366,084

1. A computer system comprising:one or more hardware processors;
system memory coupled to the one or more hardware processors, the system memory storing instructions that are executable by the one or more hardware processors;
the one or more hardware processors configured to execute the instructions stored in the system memory to pipeline result sets with fault tolerance in distributed query execution, including the following:
access a job graph, the job graph indicating a plurality of bubbles, each bubble including one or more supervertices from a plurality of supervertices, each supervertex including one or more vertices of a same vertex type, the job graph divided into the plurality of bubbles based on determined resource consumption for each of the plurality of supervertices and dependencies between supervertices within the plurality of supervertices; and
execute the job graph using resources of a distributed system including:
for a bubble in the plurality of bubbles, streaming results from one supervertex within the bubble to another supervertex within the bubble via one of:
memory or a network connection;
for another bubble in the plurality of bubbles, storing other results from a supervertex within the another bubble to durable storage; and
for a further bubble in the plurality of bubbles, accessing the other results from the durable storage.