OceanBase 分布式執(zhí)行計(jì)劃調(diào)度

2021-06-30 09:57 更新

分布式執(zhí)行計(jì)劃的簡單調(diào)度模型如下：

在執(zhí)行計(jì)劃生成的最后階段，以 EXCHANGE 節(jié)點(diǎn)為界，拆分成多個子計(jì)劃，每個子計(jì)劃被封裝成為一個 DFO，在并行度大于 1 的場景下，會一次調(diào)度兩個 DFO，依次完成 DFO 樹的遍歷執(zhí)行；在并行度等于 1 的場景下，每個 DFO 會將產(chǎn)生的數(shù)據(jù)存入中間結(jié)果管理器，按照后序遍歷的形式完成整個 DFO 樹的遍歷執(zhí)行。

單 DFO 調(diào)度

示例：在并行度為 1 的場景下，對于查詢計(jì)劃執(zhí)行單 DFO 調(diào)度。

======================================================================================
|ID|OPERATOR                             |NAME                 |EST. ROWS |COST      |
--------------------------------------------------------------------------------------
|0 |LIMIT                                |                     |10        |6956829987|
|1 | PX COORDINATOR MERGE SORT           |                     |10        |6956829985|
|2 |  EXCHANGE OUT DISTR                 |:EX10002             |10        |6956829976|
|3 |   LIMIT                             |                     |10        |6956829976|
|4 |    TOP-N SORT                       |                     |10        |6956829975|
|5 |     HASH GROUP BY                   |                     |454381562 |5815592885|
|6 |      HASH JOIN                      |                     |500918979 |5299414557|
|7 |       EXCHANGE IN DISTR             |                     |225943610 |2081426759|
|8 |        EXCHANGE OUT DISTR (PKEY)    |:EX10001             |225943610 |1958446695|
|9 |         MATERIAL                    |                     |225943610 |1958446695|
|10|          HASH JOIN                  |                     |225943610 |1480989849|
|11|           JOIN FILTER CREATE        |                     |30142669  |122441311 |
|12|            PX PARTITION ITERATOR    |                     |30142669  |122441311 |
|13|             TABLE SCAN              |CUSTOMER             |30142669  |122441311 |
|14|           EXCHANGE IN DISTR         |                     |731011898 |900388059 |
|15|            EXCHANGE OUT DISTR (PKEY)|:EX10000             |731011898 |614947815 |
|16|             JOIN FILTER USE         |                     |731011898 |614947815 |
|17|              PX BLOCK ITERATOR      |                     |731011898 |614947815 |
|18|               TABLE SCAN            |ORDERS               |731011898 |614947815 |
|19|       PX PARTITION ITERATOR         |                     |3243094528|1040696710|
|20|        TABLE SCAN                   |LINEITEM(I_L_Q06_001)|3243094528|1040696710|
======================================================================================

如下圖所示，DFO 樹除 ROOT DFO 外，在垂直方向上被分別劃分為 0、1、2 號 DFO, 從而后序遍歷調(diào)度的順序?yàn)?0->1->2，即可完成整個計(jì)劃樹的迭代。

1

兩 DFO 調(diào)度

示例：對于并行度大于 1 的計(jì)劃, 對于查詢計(jì)劃執(zhí)行兩 DFO 調(diào)度。

Query Plan
=============================================================================
|ID|OPERATOR                                   |NAME    |EST. ROWS|COST     |
-----------------------------------------------------------------------------
|0 |PX COORDINATOR MERGE SORT                  |        |9873917  |692436562|
|1 | EXCHANGE OUT DISTR                        |:EX10002|9873917  |689632565|
|2 |  SORT                                     |        |9873917  |689632565|
|3 |   SUBPLAN SCAN                            |VIEW5   |9873917  |636493382|
|4 |    WINDOW FUNCTION                        |        |29621749 |629924873|
|5 |     HASH GROUP BY                         |        |29621749 |624266752|
|6 |      HASH JOIN                            |        |31521003 |591048941|
|7 |       JOIN FILTER CREATE                  |        |407573   |7476793  |
|8 |        EXCHANGE IN DISTR                  |        |407573   |7476793  |
|9 |         EXCHANGE OUT DISTR (BROADCAST)    |:EX10001|407573   |7303180  |
|10|          HASH JOIN                        |        |407573   |7303180  |
|11|           JOIN FILTER CREATE              |        |1        |53       |
|12|            EXCHANGE IN DISTR              |        |1        |53       |
|13|             EXCHANGE OUT DISTR (BROADCAST)|:EX10000|1        |53       |
|14|              PX BLOCK ITERATOR            |        |1        |53       |
|15|               TABLE SCAN                  |NATION  |1        |53       |
|16|           JOIN FILTER USE                 |        |10189312 |3417602  |
|17|            PX BLOCK ITERATOR              |        |10189312 |3417602  |
|18|             TABLE SCAN                    |SUPPLIER|10189312 |3417602  |
|19|       JOIN FILTER USE                     |        |803481600|276540086|
|20|        PX PARTITION ITERATOR              |        |803481600|276540086|
|21|         TABLE SCAN                        |PARTSUPP|803481600|276540086|
=============================================================================

如下圖所示，DFO 樹除 ROOT DFO 外，被劃分為 3 個 DFO，調(diào)度時會先調(diào) 0 和 1 對應(yīng)的 DFO，待 0 號 DFO 執(zhí)行完畢后，會再調(diào)度 1 號和 2 號 DFO，依次迭代完成執(zhí)行。

2
以上內(nèi)容是否對您有幫助：
在線筆記
App下載
App下載

掃描二維碼

下載編程獅App

公眾號
微信公眾號

編程獅公眾號