Keyboard shortcuts

/ or ⌘/Ctrl K
Find a note
j / k
Next / previous section or linked note
h / l
Collapse or go to parent / expand or enter
e or Alt-click
Read a linked note here
o
Open focused note on its own
g g / G
First / last section or linked note
g h / g a
Home / all notes
g b / g t
Backlinks / table of contents
t
Cycle System, Light, Dark
? / Esc
Show / close this reference

Search: ↑/↓ or Ctrl N/P, Enter to open. Shortcuts pause while typing.

dask [247dc79b]

Tags: pandas, numpy

1. Flow order

  • Graph, scheduling, execution

    • constructs a flow order as a graph inside a python dict

    • assumptions: data is not modified in place, tasks do not hold the gil

      • python bindings for native interfaces may lock, these must be made sure to be released

2. Dask vs Spark

  • similar to spark
  • spark has a higher level data representation
  • dask uses numpy/pandas/sklear, spark is more mature and has sql support
  • dask allows to compute arbitrary graphs
  • spark allows computations from compositions of high level primitives or extending to rdd