Schedule

How we learned to use Python, K8s, and S3 to efficiently count data in the cloud.

In the talk, I will review the current state of the data transformation ecosystem, as well as alternative tools and promising projects that may replace dbt.

Parallel reading from Kafka topics, KRaft, server balancing, and tiered storage.

I will tell you about implementing a review and deployment process for NiFi threads in a team with many developers, where changes to the threads are made several times a day.

How we solved the problem of making hotfix changes to ETL pipelines on Apache Spark in hundreds of existing processes without changing their code.

The talk shows pitfalls that prevent the widespread use of sketches by final analysts.

In this talk, I will analyze a practical approach to measuring self-hosted LLM performance.

The talk is about practical experience in optimizing inference and ML-serving based on GPUStack in the production environment of the corporate AI Portal.

We will show how we built a scalable ML platform for detecting hackers using open-source tools (Airflow, Trino, Iceberg, and MLflow).

I will tell you how we built a single knowledge graph on top of dozens of disparate corporate datasets — an infrastructure where an AI agent doesn't guess an answer based on similar chunks, but consciously navigates the structure and relationships of data.

I'll show you how to count memory and KV-cache, how inference layer solutions change the load profile, and then we'll move on to our implementation in Deckhouse.

Let's explore how to create a good semantic search engine.

How pgvector works: vector storage, HNSW and IVFFlat algorithms, performance degradation points. An honest breakdown of where the solution holds up and where it doesn't.

I will tell you about the experience of implementing and using YTsaurus in Chestny Znak.

Let's explore what actually happens between a request and a result.

The talk focuses on practical PostgreSQL performance diagnostics for backend developers who maintain their databases independently and do not have a dedicated DBA.

Let's talk about testing, finding and debugging problems in highly loaded software, as well as support for storage with third-party vendor solutions.

I will explain how we implemented an atomic commit of distributed transactions at the PostgreSQL core level, based on the processing of 2PC/XA mechanisms, and show the results of its testing.

The talk focuses on practical experience in building a Data Streaming Lakehouse for near-real-time analytics using a stack of MySQL, Flink, Paimon, HDFS, and StarRocks.

We'll dive into the architectural decisions behind CubeFS that make it possible to build exabyte-scale storage for ML and analytics workloads. Topics include its high-performance, horizontally scalable metadata service, local and distributed caching, transparent data movement across storage tiers, and other key design features.

We are going to discuss the real experience of migrating data marts from a monolithic solution based on Greenplum 6 to the Data Lakehouse stack, paying attention to how to make this process the least painful for users. You will learn what non-obvious problems you will have to face and how to build processes so that the new architecture is more efficient than the legacy solution, rather than its less productive copy.

Let's talk about what important functions are needed to manage Iceberg tables and the role of REST Catalog in this.

Our Trino storage hit the performance ceiling of a single Ceph cluster — so we started spreading every table across several clusters at once, hiding all the sharding logic in the HAProxy sidecars on our compute nodes, without adding a single new component to the architecture. Reads sped up from 20 to 60–80 GB/s, and GET latency dropped from minutes to 1–2 seconds.

A classic MDM system often assumes that data needs to be brought together in one place: loaded, normalized, matched, assigned a golden record, and then managed centrally as master data. But what do you do when, due to security or regulatory requirements, the system is not allowed to store data within its own perimeter?

On the production pipeline, we will show how one merge triggers validation, compatibility checks, ingestion generation, data publishing, and catalog updates.

A talk about why a collection of fragmented tools stops working at the scale of a large Data Platform, and why the platform should be viewed as a unified ADLC rather than a set of separate services. I will show how this affects ETL, ad hoc development, Data Governance, Data Quality, and metrics, and why AI and the agent-based approach are becoming the main drivers of new platform requirements.

I will tell you how we built the Magnit Data ecosystem, where the catalog, glossary, DQ engine, dashboards, and chatbot work as one mechanism.

I will analyze the components of success and failure and provide a practical checklist that will help you quickly decide whether you need an agent or a classic AutoML model to generate a baseline model.

LLM agents confidently hallucinate in business reports, and the accuracy of Text-to-SQL is clearly insufficient for regulatory and management reporting. I will show you how a semantic layer based on MetricFlow can increase accuracy to 90% or higher, and how to deploy this solution on-prem to ensure that your reports can be trusted.

How Vectorless helps you deal with the problem of losing data hierarchy.