Talk

The metadata processor for data gathering and analysis

  • In Russian
Presentation pdf

Nowadays, using automatic data processing pipelines is ubiquitous. We need automatization both for local pipelines and distributed computing and map-reduce frameworks. One known problem for describing such pipelines is to define individual actions and their parameters without sharing the code on the cluster. For example, full scale heterogenous computing is quite limited. In this talk, Alexander will present a concept of "metadata processor": a way to automatically build and share the workflow between separate nodes that does not involve code sharing and allows to go into full heterogenous mode (different nodes do different tasks). As an example, he will show DataForge framework that implements this principles.

  • #валидация_конфигурации
  • #параллельные_вычисления

Speakers

Invited experts

Schedule