Article January 23, 2024

api

Words count 1.3k Reading time 1 mins. Read count 0

api.java

该包用于java编程时

JavaDoubleRDD

将scala Double 转java Double

注意一行代码:import java.lang.{Double => JDouble}
scala语法起了个类的别名

而java也有类似的,如import com.example.Calendar as MyCalendar

broadcast

keep a read-only variable cached on each machine rather than tasks

TorrentBroadcast

driver将广播对象切成small chunks
避免driver单点分发成为瓶颈,executor从driver的BlockManager取到数据后可以供其他executor获取

deploy

提交任务

rdd

CoalescedRDD

DefaultPartitionCoalescer

locality information

0%