How To Dev

Introduction Overhead When you store a text string on disk, PostgreSQL cannot simply write those raw letters. If it only did that, upon reading the data back, Postgres would not know: How many bytes long is this string? Where does it end so it can read the next data column? Is this a short string, a long string, or compressed? Therefore, Postgres must spend a few extra bytes right at the beginning of the string to record this management information. This additional consumed capacity is called Overhead, which is the accompanying management cost. Header Structure PostgreSQL uses a common format named varlena (Variable-length array) to store all string types ( TEXT, VARCHAR, BYTEA, JSONB... ). This Overhead portion varies depending on the length of the data string: Very short strings (Under 127 bytes) Overhead: 1 byte . Mechanism: Postgres uses this first 1 byte ( 8 bits ) to store 7 bits for recording the actual length of the string, and 1 bit as a flag signaling that this is a s...

Introduction In computer science and systems engineering, overhead is not a bug, but it is a phenomenon of system resource waste. Overhead is the amount of time, memory, bandwidth or CPU power consumed to manage or operate the system, rather than contributing directly to the actual data processing results. In database management systems, overhead occurs very frequently. Especially, if the system suffers from excessive memory (RAM) overhead, the Operating System (OS) will trigger a mechanism to kill that process to save the server. Common Types of Overhead in Postgres Postgres has a process-based architecture, meaning each client connection generates an independent process, so it easily encounters the following types of overhead: Connection Overhead: If you have 500 concurrent connections, Postgres will create 500 processes. The fact that the CPU must constantly context switch among these 500 processes creates a massive amount of overhead, significantly reducing performance. Memory Over...