Posts

Showing posts with the label text

Character Types

Image
Introduction Overhead When you store a text string on disk, PostgreSQL cannot simply write those raw letters. If it only did that, upon reading the data back, Postgres would not know: How many bytes long is this string? Where does it end so it can read the next data column? Is this a short string, a long string, or compressed? Therefore, Postgres must spend a few extra bytes right at the beginning of the string to record this management information. This additional consumed capacity is called Overhead, which is the accompanying management cost. Header Structure PostgreSQL uses a common format named varlena (Variable-length array) to store all string types ( TEXT, VARCHAR, BYTEA, JSONB... ). This Overhead portion varies depending on the length of the data string: Very short strings (Under 127 bytes) Overhead: 1 byte . Mechanism: Postgres uses this first 1 byte ( 8 bits ) to store 7 bits for recording the actual length of the string, and 1 bit as a flag signaling that this is a s...

GIN Index with TEXT

Image
Introduction In this article, we will continue to explore how to use GIN Index with the text data type First, please note that GIN Index only works with multi-valued data types, meaning fields containing multiple values, so you can easily create an index with the array type However, if you create a similar index with the text type, an error will be reported because text is only a single-valued type, containing only one content string, so you must pass an additional function that defines how to split the text value into items in an array before the index can be created If you remember, when using GIN Index with a text array, you will encounter limitations regarding partial search and must enter the exact word to search for it to work, now you can solve that problem by using text combined with two functions, to_tsvector and pg_trgm Full-Text Search This method uses the to_tsvector function and its index creation process will be as follows Tokenization & Normalization Split the text...