How To Dev

Introduction A distribution system is a collection of independent computers in terms of hardware but communicating and coordinating with each other through a computer network, making end users feel like a single centralized system. In the era of big data and cloud computing, distributed architecture plays a backbone role for almost all large-scale online services. Core Characteristics For a system to be considered truly distributed, it must face and solve the following technical characteristics: No Shared Clock : Each node has its own physical clock and there is always a certain clock drift. Therefore, you cannot rely on absolute real time to determine the order of events happening between nodes. No Shared Memory : Nodes cannot directly read or write to each other's RAM. Every communication activity and state synchronization must be done through message passing such as protocols like HTTP/REST, gRPC or message brokers like Kafka, RabbitMQ. Concurrency : Multiple nodes process indep...

Introduction In the era of big data, when storage capacity reaches the Terabyte threshold or the number of requests exceeds the physical limit of a single server ( Vertical Scaling / Scale-up ), Sharding is the optimal solution for Horizontal Scaling / Scale-out It is achieved by dividing a massive data table (billions of rows) into multiple small, independent and self-managed parts called Shards Each Shard is a separate physical database located on a different physical Server . The key point is that the data in the Shards does not overlap, but when combined, it forms a complete dataset. Horizontal Partitioning vs Sharding Horizontal Partitioning : Splitting a large table into smaller tables (such as by month) but all of these child tables still reside on the same physical server. Sharding : Distributing those child tables after partitioning across multiple different physical servers. Therefore, Sharding is the architecture of horizontal data partitioning in a distributed environ...