Implementing Sharding with Citus
Introduction In the era of big data, when storage capacity reaches the Terabyte threshold or the number of requests exceeds the physical limit of a single server ( Vertical Scaling / Scale-up ), Sharding is the optimal solution for Horizontal Scaling / Scale-out It is achieved by dividing a massive data table (billions of rows) into multiple small, independent and self-managed parts called Shards Each Shard is a separate physical database located on a different physical Server . The key point is that the data in the Shards does not overlap, but when combined, it forms a complete dataset. Horizontal Partitioning vs Sharding Horizontal Partitioning : Splitting a large table into smaller tables (such as by month) but all of these child tables still reside on the same physical server. Sharding : Distributing those child tables after partitioning across multiple different physical servers. Therefore, Sharding is the architecture of horizontal data partitioning in a distributed environ...