Tuning query and Query Optimization

Introduction

Core Concepts

Optimizer

  • This is the general name for the entire component responsible for receiving SQL statements, analyzing them and deciding how the system will retrieve the data.
  • Its task is to transform a declarative SQL statement (such as what data to get) into a specific physical execution plan (how to get that data step by step).
  • This is the most general concept that most current databases like SQL or NoSQL use to automatically calculate the most efficient way to execute a query.

Cost-based Optimizer

  • This is a specific type of Optimizer that operates based on the principle of Cost calculation. It helps the Optimizer choose the most efficient execution path.
  • To see the difference clearly, there are two main schools of Optimizer in computer science history:
    • Rule-based Optimizer (RBO): Chooses the path based on fixed, rigid rules (For example: if an Index is available, it must be used, regardless of table size), which lacks flexibility and is now obsolete.
    • Cost-based Optimizer (CBO): Chooses the path by calculating and comparing the Cost of thousands of different ways. The path with the lowest Cost is used. Most modern databases like PostgreSQL, MySQL, Oracle and SQL Server use this CBO format today.

Query Planner Cost Model

  • If the CBO is used to select the cheapest option, the Query Planner Cost Model is the core system of formulas used to provide cost details for each action to calculate.
  • Task: This is a set of constants, parameters and mathematical formulas pre-programmed into the Database source code used to price each data read/write action.
  • Real-world example in PostgreSQL: The Cost Model regulates basic costs such as:
    • seq_page_cost = 1.0 (The cost to read 1 data page sequentially from the hard drive).
    • random_page_cost = 4.0 (The cost for Random I/O to read 1 data page randomly from the hard drive - more expensive than sequential reading).
    • cpu_tuple_cost = 0.01 (The cost for the CPU to process 1 row of data).
  • When you write a query, the CBO will take these constants from the Cost Model, multiply them by the current number of rows to get the total Cost figure commonly seen in the EXPLAIN command.

Tuning query

  • Optimizing data queries is the process of improving the speed and performance of SQL statements (such as SELECT, UPDATE, DELETE) so that the system responds faster, consuming the least CPU, RAM and hard drive (I/O) resources possible.
  • When data expands to millions of records, an unoptimized SQL statement can turn a millisecond action into several minutes, clogging the entire system.

Query Optimization

While Query Tuning is the manual modification of statements by hand, Query Optimization is the entire process where the Database Engine automatically analyzes, calculates and finds the shortest, least resource-intensive path to execute that statement.

How it works

  • Most modern database management systems (MySQL, PostgreSQL, Oracle, SQL Server) use an optimization engine called Cost-Based Optimizer (CBO).
  • When you execute an SQL statement, it will not run immediately but must go through a process as follows:
    • Parsing: Analyzes and checks if the statement is written in the correct SQL syntax and whether the tables and columns you call exist in the Database.
    • Rewriting: The optimizer will automatically rewrite your statement in a smarter way without changing the results (such as converting Subqueries to JOIN operations if deemed more efficient).
    • Optimization: The system will generate a series of different Execution Plans.
      • It relies on Statistics data stored in the Database (such as the number of rows in the table, data dispersion, density of duplicate values) to calculate an index called Cost for each plan.
      • This cost is usually calculated by the level of I/O utilization, which is the number of Pages to be read from disk and the CPU resources to be consumed.
  • Execution: The plan with the lowest Cost will be selected and sent down to the Storage Engine to retrieve the data.

Limitations

Despite being very smart, the Optimizer can still sometimes choose a terrible execution plan (causing the query to run for a long time). Common reasons include:

  • Outdated Statistics data
    • If your table suddenly jumps from 1,000 rows to 10,000,000 rows but the Database has not updated the statistics, the Optimizer still thinks this table is very small. It may decide to do a Full Table Scan instead of using an Index, impacting performance.
    • The solution is to run a manual statistics update command so the Database recognizes table information using the ANALYZE table_name command.
  • Complex Logic
    • When you JOIN too many tables (usually over 12 tables), the number of feasible plans increases exponentially. The Optimizer will not be able to calculate all cases due to timeout.
    • It is forced to activate the Genetic Query Optimizer (GEQO) algorithm to solve the Combinatorial Explosion.
    • At this point, it only generates a Local Optimum, which is a result "Good enough in an acceptable time", rather than committing to finding the Global Optimum, the absolute best Plan.
    • Solution
      • For other Databases like MySQL, you can use Index Hints / Optimizer Hints to force the Optimizer to choose the Index or algorithm you want.
      • On Postgres, this can also be done but it is not supported by default, so you should use CTEs instead to split the JOIN of many tables into multiple stages, as joining a smaller number of tables will help the Optimizer operate efficiently and accurately without needing GEQO.

Query Tuning and Query Optimization

Thus, the difference is as follows

  • Query Tuning is performed by Developers/Database Admins to rewrite SQL code, create more Indexes, split or change table structures to provide an explicit statement and good data structure to suggest efficient execution for the Database.
  • Query Optimization is the job of the Database Engine, where the Optimizer analyzes the statement, calculates the Cost, chooses the available Index and appropriate JOIN algorithm to find the fastest Physical Plan based on the received statement.

Detail

Before Tuning, you need to identify which queries have problems that need Optimization. Postgres supports logging query history, which you can configure as follows:

Open the postgresql.conf file and edit the shared_preload_libraries field as follows:

shared_preload_libraries = 'pg_stat_statements'

Restart Postgres and you can use the pg_stat_statements Extension:

CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

-- Query 1
SELECT 
    query,
    calls AS execution_count,
    round(mean_exec_time::numeric, 2) AS average_time_ms,
    round(max_exec_time::numeric, 2) AS max_time_ms
FROM pg_stat_statements
ORDER BY mean_exec_time DESC
LIMIT 5;

-- Query 2
SELECT 
    query,
    calls AS execution_count,
    round(total_exec_time::numeric, 2) AS total_time_ms,
    round(mean_exec_time::numeric, 2) AS average_time_ms,
    round((100 * total_exec_time / sum(total_exec_time) OVER())::numeric, 2) AS resource_percentage
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 5;
  • Query 1: Top 5 statements with the largest average execution time
  • Query 2: Top 5 statements occupying the most total execution time, calculated by the average execution time of 1 run multiplied by the number of runs


This query is used to view running statements sorted by execution time, used when a query is running too long and you need to stop it immediately:

SELECT 
    pid,
    usename AS database_user, 
    age(clock_timestamp(), query_start) AS running_duration,
    query AS sql_statement,
    state AS execution_state
FROM pg_stat_activity
WHERE state != 'idle' AND query NOT LIKE '%pg_stat_activity%'
ORDER BY running_duration DESC;

SELECT pg_cancel_backend(39);

Please check with the following 2 Transactions:

-- Transaction 1
BEGIN;
SELECT * FROM products FOR UPDATE;

-- Transaction 2
BEGIN;
UPDATE products
SET name = 'New name'
WHERE id = 'ea2ba4e5-88e9-4397-8352-32ce6b301708';
  • Transaction 1 will create a lock and Transaction 2 will be blocked
  • Combined with the Query above, you will see the information of Transaction 2 along with the PID
  • Then you can use the pg_cancel_backend function to kill that Transaction




The following are ways to clean up and analyze a query:

-- Statement 1
ANALYZE

-- Statement 2
ANALYZE products;
ANALYZE public."products";

-- Statement 3
VACUUM ANALYZE public."products";

-- Statement 4
EXPLAIN {query}

-- Statement 5
EXPLAIN ANALYZE {query}

-- Statement 6
EXPLAIN (ANALYZE, COSTS, VERBOSE, BUFFERS, FORMAT JSON) {query}

-- Query
SELECT 
    t.name AS tenant_name,
    o.id AS order_id,
    o.customer_name,
    o.total_amount,
    o.status AS order_status,
    o.created_at AS order_date
FROM orders o
JOIN tenants t ON o.tenant_id = t.id
JOIN products p ON p.tenant_id = t.id 
WHERE 
    t.id = '36321616-093d-491e-b9b3-49256f59585c'
    AND o.status = 'pending'
    AND o.created_at >= NOW() - INTERVAL '30 days'
GROUP BY t.name, o.id, o.customer_name, o.total_amount, o.status, o.created_at;
  • Statement 1: Scans through all tables in the current database to collect and update statistics data.
  • Statement 2: Works exactly like the command above, but focuses only on 1 table products
    • You can choose to write explicitly to specify the schema and table as public."products"
    • Or just write the table name as products then the default is using the public schema
  • Statement 3: Performs two consecutive actions on the products table:
    • VACUUM: Due to the MVCC mechanism of Postgres, when you UPDATE or DELETE data, old rows are not deleted immediately from the hard drive but turn into Dead Tuples. The VACUUM command will scan the table, marking the memory regions containing these dead rows as empty so Postgres can reuse them for new data rows later, avoiding Table Bloat.
    • ANALYZE: After cleaning up, it immediately updates the latest statistics of the table for the Optimizer.
  • Statement 4: Predicts the execution plan.
    • How it works: PostgreSQL only parses and provides estimates on cost, rows returned and width of the row.
    • Characteristics: Does not run the actual SQL statement. Very safe when checking heavy DELETE, UPDATE or INSERT statements because it does not change data.
  • Statement 5: Measures actual performance.
    • How it works: PostgreSQL will execute that SQL statement, then display both parameters: estimated data (like Statement 1) and actual data (actual time, actual rows, loops).
    • Characteristics: Helps you compare whether the database estimates are accurate to find the cause of a slow query.
    • Note that this statement will actually run, if you use it with UPDATE/DELETE then the data in the Database will be changed for real. You should wrap it in a ROLLBACK Transaction if you only want to test performance.
  • Statement 6: Gets maximum details and exports structured data, specifically as follows:
    • ANALYZE: Runs the query to get actual parameters.
    • COSTS: Displays estimated costs (available by default, written to be explicit).
    • VERBOSE: Displays additional secondary information such as the list of columns selected at each step, table schema.
    • BUFFERS: Displays the number of data blocks read/written from RAM (shared hit) or from disk (read/written). This is an extremely important parameter for I/O optimization.
    • FORMAT JSON: Exports returned results under JSON format (instead of the usual text directory tree format). This format helps visualization tools like Postgres Explain Visualizer analyze and draw visual charts.







This is the JSON content describing in detail how Postgres executes the statement:

[
  {
    "Plan": {
      "Node Type": "Aggregate",
      "Strategy": "Hashed",
      "Partial Mode": "Simple",
      "Parallel Aware": false,
      "Async Capable": false,
      "Startup Cost": 72.78,
      "Total Cost": 72.99,
      "Plan Rows": 21,
      "Plan Width": 61,
      "Actual Startup Time": 0.209,
      "Actual Total Time": 0.214,
      "Actual Rows": 21.00,
      "Actual Loops": 1,
      "Disabled": false,
      "Output": ["t.name", "o.id", "o.customer_name", "o.total_amount", "o.status", "o.created_at"],
      "Group Key": ["t.name", "o.id"],
      "Planned Partitions": 0,
      "HashAgg Batches": 1,
      "Peak Memory Usage": 32,
      "Disk Usage": 0,
      "Shared Hit Blocks": 28,
      "Shared Read Blocks": 0,
      "Shared Dirtied Blocks": 0,
      "Shared Written Blocks": 0,
      "Local Hit Blocks": 0,
      "Local Read Blocks": 0,
      "Local Dirtied Blocks": 0,
      "Local Written Blocks": 0,
      "Temp Read Blocks": 0,
      "Temp Written Blocks": 0,
      "Plans": [
        {
          "Node Type": "Nested Loop",
          "Parent Relationship": "Outer",
          "Parallel Aware": false,
          "Async Capable": false,
          "Join Type": "Inner",
          "Startup Cost": 4.79,
          "Total Cost": 71.94,
          "Plan Rows": 168,
          "Plan Width": 61,
          "Actual Startup Time": 0.097,
          "Actual Total Time": 0.166,
          "Actual Rows": 168.00,
          "Actual Loops": 1,
          "Disabled": false,
          "Output": ["t.name", "o.id", "o.customer_name", "o.total_amount", "o.status", "o.created_at"],
          "Inner Unique": false,
          "Shared Hit Blocks": 28,
          "Shared Read Blocks": 0,
          "Shared Dirtied Blocks": 0,
          "Shared Written Blocks": 0,
          "Local Hit Blocks": 0,
          "Local Read Blocks": 0,
          "Local Dirtied Blocks": 0,
          "Local Written Blocks": 0,
          "Temp Read Blocks": 0,
          "Temp Written Blocks": 0,
          "Plans": [
            {
              "Node Type": "Bitmap Heap Scan",
              "Parent Relationship": "Outer",
              "Parallel Aware": false,
              "Async Capable": false,
              "Relation Name": "orders",
              "Schema": "public",
              "Alias": "o",
              "Startup Cost": 4.51,
              "Total Cost": 63.08,
              "Plan Rows": 21,
              "Plan Width": 68,
              "Actual Startup Time": 0.061,
              "Actual Total Time": 0.098,
              "Actual Rows": 21.00,
              "Actual Loops": 1,
              "Disabled": false,
              "Output": ["o.id", "o.tenant_id", "o.customer_name", "o.total_amount", "o.status", "o.created_at"],
              "Recheck Cond": "((o.tenant_id = '36321616-093d-491e-b9b3-49256f59585c'::uuid) AND ((o.status)::text = 'pending'::text))",
              "Rows Removed by Index Recheck": 0,
              "Filter": "(o.created_at >= (now() - '30 days'::interval))",
              "Rows Removed by Filter": 1,
              "Exact Heap Blocks": 22,
              "Lossy Heap Blocks": 0,
              "Shared Hit Blocks": 24,
              "Shared Read Blocks": 0,
              "Shared Dirtied Blocks": 0,
              "Shared Written Blocks": 0,
              "Local Hit Blocks": 0,
              "Local Read Blocks": 0,
              "Local Dirtied Blocks": 0,
              "Local Written Blocks": 0,
              "Temp Read Blocks": 0,
              "Temp Written Blocks": 0,
              "Plans": [
                {
                  "Node Type": "Bitmap Index Scan",
                  "Parent Relationship": "Outer",
                  "Parallel Aware": false,
                  "Async Capable": false,
                  "Index Name": "idx_orders_tenant_status",
                  "Startup Cost": 0.00,
                  "Total Cost": 4.50,
                  "Plan Rows": 22,
                  "Plan Width": 0,
                  "Actual Startup Time": 0.038,
                  "Actual Total Time": 0.038,
                  "Actual Rows": 22.00,
                  "Actual Loops": 1,
                  "Disabled": false,
                  "Output": [],
                  "Index Cond": "((o.tenant_id = '36321616-093d-491e-b9b3-49256f59585c'::uuid) AND ((o.status)::text = 'pending'::text))",
                  "Index Searches": 1,
                  "Shared Hit Blocks": 2,
                  "Shared Read Blocks": 0,
                  "Shared Dirtied Blocks": 0,
                  "Shared Written Blocks": 0,
                  "Local Hit Blocks": 0,
                  "Local Read Blocks": 0,
                  "Local Dirtied Blocks": 0,
                  "Local Written Blocks": 0,
                  "Temp Read Blocks": 0,
                  "Temp Written Blocks": 0
                }
              ]
            },
            {
              "Node Type": "Materialize",
              "Parent Relationship": "Inner",
              "Parallel Aware": false,
              "Async Capable": false,
              "Startup Cost": 0.28,
              "Total Cost": 6.79,
              "Plan Rows": 8,
              "Plan Width": 25,
              "Actual Startup Time": 0.002,
              "Actual Total Time": 0.002,
              "Actual Rows": 8.00,
              "Actual Loops": 21,
              "Disabled": false,
              "Output": ["t.name", "t.id"],
              "Storage": "Memory",
              "Maximum Storage": 17,
              "Shared Hit Blocks": 4,
              "Shared Read Blocks": 0,
              "Shared Dirtied Blocks": 0,
              "Shared Written Blocks": 0,
              "Local Hit Blocks": 0,
              "Local Read Blocks": 0,
              "Local Dirtied Blocks": 0,
              "Local Written Blocks": 0,
              "Temp Read Blocks": 0,
              "Temp Written Blocks": 0,
              "Plans": [
                {
                  "Node Type": "Nested Loop",
                  "Parent Relationship": "Outer",
                  "Parallel Aware": false,
                  "Async Capable": false,
                  "Join Type": "Inner",
                  "Startup Cost": 0.28,
                  "Total Cost": 6.75,
                  "Plan Rows": 8,
                  "Plan Width": 25,
                  "Actual Startup Time": 0.025,
                  "Actual Total Time": 0.032,
                  "Actual Rows": 8.00,
                  "Actual Loops": 1,
                  "Disabled": false,
                  "Output": ["t.name", "t.id"],
                  "Inner Unique": false,
                  "Shared Hit Blocks": 4,
                  "Shared Read Blocks": 0,
                  "Shared Dirtied Blocks": 0,
                  "Shared Written Blocks": 0,
                  "Local Hit Blocks": 0,
                  "Local Read Blocks": 0,
                  "Local Dirtied Blocks": 0,
                  "Local Written Blocks": 0,
                  "Temp Read Blocks": 0,
                  "Temp Written Blocks": 0,
                  "Plans": [
                    {
                      "Node Type": "Seq Scan",
                      "Parent Relationship": "Outer",
                      "Parallel Aware": false,
                      "Async Capable": false,
                      "Relation Name": "tenants",
                      "Schema": "public",
                      "Alias": "t",
                      "Startup Cost": 0.00,
                      "Total Cost": 2.25,
                      "Plan Rows": 1,
                      "Plan Width": 25,
                      "Actual Startup Time": 0.008,
                      "Actual Total Time": 0.013,
                      "Actual Rows": 1.00,
                      "Actual Loops": 1,
                      "Disabled": false,
                      "Output": ["t.id", "t.name", "t.created_at"],
                      "Filter": "(t.id = '36321616-093d-491e-b9b3-49256f59585c'::uuid)",
                      "Rows Removed by Filter": 99,
                      "Shared Hit Blocks": 1,
                      "Shared Read Blocks": 0,
                      "Shared Dirtied Blocks": 0,
                      "Shared Written Blocks": 0,
                      "Local Hit Blocks": 0,
                      "Local Read Blocks": 0,
                      "Local Dirtied Blocks": 0,
                      "Local Written Blocks": 0,
                      "Temp Read Blocks": 0,
                      "Temp Written Blocks": 0
                    },
                    {
                      "Node Type": "Index Only Scan",
                      "Parent Relationship": "Inner",
                      "Parallel Aware": false,
                      "Async Capable": false,
                      "Scan Direction": "Forward",
                      "Index Name": "idx_products_tenant",
                      "Relation Name": "products",
                      "Schema": "public",
                      "Alias": "p",
                      "Startup Cost": 0.28,
                      "Total Cost": 4.42,
                      "Plan Rows": 8,
                      "Plan Width": 16,
                      "Actual Startup Time": 0.016,
                      "Actual Total Time": 0.017,
                      "Actual Rows": 8.00,
                      "Actual Loops": 1,
                      "Disabled": false,
                      "Output": ["p.tenant_id"],
                      "Index Cond": "(p.tenant_id = '36321616-093d-491e-b9b3-49256f59585c'::uuid)",
                      "Rows Removed by Index Recheck": 0,
                      "Heap Fetches": 0,
                      "Index Searches": 1,
                      "Shared Hit Blocks": 3,
                      "Shared Read Blocks": 0,
                      "Shared Dirtied Blocks": 0,
                      "Shared Written Blocks": 0,
                      "Local Hit Blocks": 0,
                      "Local Read Blocks": 0,
                      "Local Dirtied Blocks": 0,
                      "Local Written Blocks": 0,
                      "Temp Read Blocks": 0,
                      "Temp Written Blocks": 0
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    "Query Identifier": 8645693554078660638,
    "Planning": {
      "Shared Hit Blocks": 0,
      "Shared Read Blocks": 0,
      "Shared Dirtied Blocks": 0,
      "Shared Written Blocks": 0,
      "Local Hit Blocks": 0,
      "Local Read Blocks": 0,
      "Local Dirtied Blocks": 0,
      "Local Written Blocks": 0,
      "Temp Read Blocks": 0,
      "Temp Written Blocks": 0
    },
    "Planning Time": 0.254,
    "Triggers": [
    ],
    "Execution Time": 0.329
  }
]

The execution order of the statement will be from bottom to top (inside to outside) with specific steps as follows:

  • Scan and filter the tenants (t) table
    • Performs a Seq Scan on the tenants table.
    • Filters to find rows with t.id = '36321616-093d-491e-b9b3-49256f59585c'.
    • The result obtained is 1 row matching the condition after removing 99 non-matching rows.
  • Use Index of the products (p) table
    • Uses an Index Only Scan idx_products_tenant.
    • With Index Cond as p.tenant_id = '36321616-093d-491e-b9b3-49256f59585c'.
    • The result is 8 product data rows. Since this is an Index Only Scan, data is retrieved directly from the Index without accessing the table disk pages (Heap Fetches: 0).
  • Inner Join between tenants and products
    • Uses the Nested Loop algorithm.
    • Takes 1 row result from Step 1 as the outer loop (Outer), matches against 8 rows found from Step 2 in the inner loop (Inner).
    • Creates a combination of 8 data rows containing tenant and product information.
  • Temporarily save the results to Materialize
    • This retention helps PostgreSQL quickly reuse the dataset (tenant + products) in subsequent loops without running each step from the beginning.
    • "Maximum Storage": 17 represents a maximum memory consumption of only 17 bytes.
  • Search for orders in the orders (o) table
    • This takes place in parallel with joining tenants and products
    • Finds orders matching the conditions using a Bitmap Index Scan based on the Index idx_orders_tenant_status
    • Filters by tenant_id and o.status = 'pending'
    • Uses a Bitmap Heap Scan to fetch data directly from the Heap, then applies an additional filter for o.created_at >= NOW() - INTERVAL '30 days'
    • The result contains 21 valid order rows, removing 1 outdated row
  • Use Nested Loop Inner Join between the tenants-products data cluster and orders
    • Because the filters have already been applied, it only needs to multiply these two datasets: the tenants-products cluster (8 rows) x orders (21 rows)
    • The result is 8 x 21 = 168 rows ("Actual Rows": 168.00)
  • Use Hashed Aggregate to process the GROUP BY
    • Grouping is required based on t.name, o.id, o.customer_name, o.total_amount, o.status, o.created_at for 168 rows
    • Since these orders all share the same tenant_id, the result will be the original 21 rows retrieved from the orders table
  • Performance:
    • Planning time: 0.254 ms
    • Execution time: 0.329 ms

Happy coding!

See more articles here.

Comments

Post a Comment

Popular posts from this blog

All Practice Series

Image
Introduction This is a comprehensive page about the technologies I have shared in series format. You can view brief introductions and links to directly access each series you are interested in. In the field of software development, to deploy a product from the initial idea to its release, the standard process typically involves several stages as follows: Database : Designing and implementing the database according to business requirements, storing data during the system's operation. Backend : Handling the main logic of the system, communicating with the database and services. Frontend : Building the interface for users to interact with the system, which could be a desktop, mobile, or web application. This usually includes implementing UI/UX and integrating APIs from the backend. DevOps : Deploying the system for use, which can be done on a server or in the cloud. Testing : Applying testing methods to ensure the product meets the standards for release. Of course, these are just stan...
Read more

Kubernetes Deployment for Zero Downtime

Image
Introduction In Kubernetes (K8s) , a Pod is the smallest resource unit used to run one or more containers during deployment. There are several ways to create a Pod : you can create it directly, use a ReplicationController , or a ReplicaSet . However, the most commonly used resource for managing Pods is the Deployment . When you use a Deployment , it actually creates a ReplicaSet to manage the Pods but comes with many additional benefits that support the deployment process. Some Advantages of Deployment : Ensures Pod Availability : It guarantees that the specified number of Pods are always running according to the configuration, automatically deploying additional Pods if any failures occur. Supports Restart and Undo Deployment : Allows you to easily restart or roll back to previous versions of your Deployment . Zero Downtime Deployment : When updating configurations or scaling Deployments , zero downtime is crucial. This means that new Pods are created while the old Pods are stil...
Read more

Deploying a NodeJS Server on Google Kubernetes Engine

Image
Introduction to GKE Google Kubernetes Engine (GKE) is a managed Kubernetes service provided by Google Cloud Platform , facilitating simple and efficient deployment of Docker images. We only need to provide some configuration for the number of nodes, machine types, and replicas to use. Some Concepts Cluster A Cluster is a collection of Nodes where Kubernetes can deploy applications. A cluster includes at least one Master Node and multiple Worker Nodes . The Master Node is used to manage the Worker Nodes . Node A Node is a server in the Kubernetes Cluster. Nodes can be physical servers or virtual machines. Each Node runs  Kubernetes , which is responsible for communication between the Master Node and Worker Node , as well as managing Pods and containers running on it. Pod A Pod is the smallest deployable unit in Kubernetes . Each Pod contains one or more containers, typically Docker containers. Containers in the same Pod share a network namespace, meaning they have the ...
Read more

Sitemap

Image
A page aggregator designed for easy reference and search, compiling all blog posts for convenient access. Loading all posts...
Read more

React Practice Series

Image
Introduction React is a JavaScript library created by Facebook , often referred to as the most popular frontend framework today. This page aims to gather articles related to ReactJS , covering topics such as theory, features, and commonly used packages in the process of building ReactJS applications. I will update this series with more articles in the future as new ideas for content come up. The articles are arranged in increasing order of difficulty, so if you have time, it's recommended to start from the beginning of the series. This will ensure you grasp the essential knowledge and information needed for the subsequent articles. Here are some key topics in the series that you need to explore to effectively use ReactJS : Fundamental : React Hook, React Reconciliation. State management : redux, mobx, recoil, zustand, xstate, etc. Middleware libraries : redux-thunk, redux-saga, redux-observable, etc. Popular packages : tasnstack query, immer, styled-components, etc. Rendering te...
Read more

Docker Practice Series

Image
Introduction Docker is an open platform for developing, shipping, and running applications. This page is dedicated to compiling articles related to Docker , covering both the theoretical aspects and practical applications of Docker in setting up popular tools essential for software development. I will be updating this series with more articles in the future as new ideas for topics arise. The articles are arranged in increasing order of difficulty, making it easier for you to follow along. If you have the time, I recommend starting from the beginning of the series to ensure that you grasp the necessary knowledge and information needed for the more advanced articles later on. Here are some key topics in the series that you need to explore to effectively use Docker : Basic knowledge Docker commands Docker Compose Building Docker images Performance improvement Integration with CI/CD Once you have a solid understanding of the foundational knowledge, the extended topics (including advanced...
Read more

Helm for beginer - Deploy nginx to Google Kubernetes Engine

Image
Introduction Helm is a package manager for Kubernetes , which simplifies the process of deploying and managing applications on Kubernetes clusters. Helm uses a packaging format called charts , which are collections of files that describe a related set of Kubernetes resources. Key Components of Helm Charts : Helm packages are called charts. A chart is a collection of files that describe a related set of Kubernetes resources. A single chart might be used to deploy something simple, like a memcached pod, or something complex, like a full web app stack with HTTP servers , databases, caches, and so on. Values : Charts can be customized with values, which are configuration settings that specify how the chart should be installed on the cluster. These values can be set in a ` values.yaml ` file or passed on the command line. Releases : When you install a chart, a new release is created. This means that one chart can be installed multiple times into the same cluster, and each can be indep...
Read more

A Handy Guide to Using Dynamic Import in JavaScript

Image
Introduction In JavaScript, when we import a file, the process usually happens synchronously, known as static import. However, as our applications grow, this synchronous loading can lead to slower initial page loads due to larger JavaScript bundles. Moreover, there are times when imports are necessary only under specific circumstances, leading to unnecessary loading times for users who might not even utilize those features. Details Instead of burdening all users with unnecessary imports, dynamic imports come to the rescue. They allow us to load modules or files only when they are needed, thus improving performance and user experience. Dynamic imports are invoked using the import() function and return a promise. When using default exports, the exported data can be accessed through the default field, while other data can be accessed through fields with matching names. Here's how you can use dynamic imports: import ( "ramda" ). then ( module => { const moduleDefaul...
Read more

DevOps Practice Series

Image
Introduction DevOps is a combination of development (Dev) and operations (Ops), aimed at uniting people, processes, and technology to enhance the software development lifecycle. Here are some key aspects of DevOps: Collaboration and Communication: DevOps fosters a culture where development, IT operations, quality engineering, and security teams work together seamlessly. Continuous Integration and Continuous Delivery (CI/CD): These practices automate the integration and delivery of code changes, ensuring faster and more reliable software releases. Infrastructure as Code (IaC): This approach involves managing and provisioning computing infrastructure through machine-readable scripts, rather than physical hardware configuration. Monitoring and Logging: Continuous monitoring and logging help teams to detect issues early and maintain system reliability. Automation: Automating repetitive tasks reduces errors and increases efficiency, allowing teams to focus on more strategic work. By impleme...
Read more

Kubernetes Practice Series

Image
Introduction Kubernetes (often abbreviated as K8s ) is an open-source system designed to automate the deployment, scaling, and management of containerized applications. This page serves as a collection of articles related to Kubernetes (K8s) , including theoretical concepts and practical guides on using Kubernetes to set up essential tools for the software development process. I will continue to update this series with new articles as ideas come to mind, to ensure the series becomes more comprehensive. The articles are arranged in increasing order of difficulty to make it easier for you to follow. If you have time, it's recommended that you start from the beginning of the series to acquire the necessary knowledge and information that will prepare you for the subsequent articles. Key Topics Covered in This Series Basic Knowledge : Fundamental concepts, common commands, etc. Resources : Pod, Deployment, Service, StatefulSet, Ingress, etc. Pod Autoscaler : Horizontal and Vertical sc...
Read more