How To Dev

Introduction In the previous article, I provided a basic guide on how to use @snaplet/seed to seed data quickly. In this article, I will show you how to generate a large amount of data in a short time, approximately 1,000,000 records, which is useful when you need to perform performance testing. You can apply a similar approach to larger datasets like several million records, using a streaming mechanism to avoid Out of Memory (OOM) errors caused by allocating too much memory at once and overloading the system. I will explain two different approaches here: Coding: chunking data and writing each small part to the database Database: creating a CSV file and using a command to copy that data directly into PostgreSQL Prerequisites You need to set up Prisma, PostgreSQL and @snaplet/seed before continuing, you can check out the previous articles for instructions. Detail First, let us create the file prisma/seed/seed.ts import {faker} from '@faker-js/faker' import {createSeedClient}...

Introduction pg-promise is a powerful library for Node.js dedicated to interacting with PostgreSQL databases. This library is built on top of the native pg driver but provides a high-level abstraction layer with a Promise-based architecture, making the source code cleaner, easier to read and more maintainable. Advantages Automatically manages connections and transactions safely Features a built-in powerful SQL formatting system to prevent SQL Injection attacks Supports handling external SQL files via Query Files and offers exceptionally high performance for bulk data insertion thanks to optimized utilities like pgp.helpers.insert and pgp.helpers.update . Prerequisites In the previous article, I provided instructions on using Prisma to seed data, but using ORMs always introduces limitations such as lack of support for all data types as well as advanced database features. Since ORMs are designed to work with various database types, they sacrifice specific functionalities to solve gener...