The metadata processor for data gathering and analysis

Nowadays, using automatic data processing pipelines is ubiquitous. We need automatization both for local pipelines and distributed computing and map-reduce frameworks. One known problem for describing such pipelines is to define individual actions and their parameters without sharing the code on the cluster. For example, full scale heterogenous computing is quite limited. In this talk, Alexander will present a concept of "metadata processor": a way to automatically build and share the workflow between separate nodes that does not involve code sharing and allows to go into full heterogenous mode (different nodes do different tasks). As an example, he will show DataForge framework that implements this principles.