Coin163

首页 > spark SQL源码阅读002——sql.core包核心类——002执行SQL语法2次解析SQL词(analyse)

spark SQL源码阅读002——sql.core包核心类——002执行SQL语法2次解析SQL词(analyse)

相关标签: spark 源码 sql

2020腾讯云双十一活动,全年最低!!!(领取3500元代金券),
地址https://cloud.tencent.com/act/cps/redirect?redirect=1074

【阿里云】双十一活动,全年抄底价,限时3天!(老用户也有),
入口地址https://www.aliyun.com/1111/home


 @DeveloperApi
  protected[sql] class <span style="font-size:18px;color:#ff99ff;">QueryExecution</span>(val logical: LogicalPlan) {
    <span style="font-size:18px;color:#ff99ff;"><strong>def assertAnalyzed(): Unit = checkAnalysis(analyzed)<span style="white-space:pre">	</span>//1.1 analyse</strong></span>

   <strong><span style="color:#ff99ff;"> <span style="font-size:18px;">lazy val analyzed: LogicalPlan = analyzer(logical)<span style="white-space:pre">		</span>//1.2 analyse
</span></span></strong>
    lazy val withCachedData: LogicalPlan = {
      assertAnalyzed
      cacheManager.useCachedData(analyzed)
    }
    lazy val optimizedPlan: LogicalPlan = optimizer(withCachedData)<span style="white-space:pre">			</span>//2 :<span style="font-family: Arial, Helvetica, sans-serif;">optimizer 优化</span>


    // TODO: Don't just pick the first one...
    lazy val sparkPlan: SparkPlan = {<span style="white-space:pre">							</span>//3: 转化成spark plan。真正在spark上跑的任务
      SparkPlan.currentContext.set(self)
      planner(optimizedPlan).next()
    }
    // executedPlan should not be used to initialize any SparkPlan. It should be
    // only used for execution.
    lazy val executedPlan: SparkPlan = prepareForExecution(sparkPlan)

    /** Internal version of the RDD. Avoids copies and has no schema */
    lazy val toRdd: RDD[Row] = executedPlan.execute()


<span style="font-size: 18px; color: rgb(255, 153, 255);">QueryExecution</span>
由collect()方法可以看到,上面的类是spark开始执行的入口类。

第一步是先对,SparkContext.sql()方法返回的简单解析过的SQL先做2次 analyse。时间太晚、明天再来

原文

@DeveloperApi protected[sql] class <span style="font-size:18px;color:#ff99ff;">QueryExecution</span>(val logical: LogicalPlan) { <span style="font-size:18px;color:#ff99ff;"><

------分隔线----------------------------
相关推荐