Module 3: Node Package Manager (NPM / PNPM / Yarn)

Package Management

Node.js projects rely on package managers to resolve, download, verify, and organize third-party libraries within the node_modules directory.

package.json vs. package-lock.json

File	Purpose	Key Contents	Version Control
package.json	Project manifest describing metadata, commands, and dependency constraints.	Names, scripts, loose semver ranges (dependencies, devDependencies, peerDependencies).	Commit to Git.
package-lock.json (yarn.lock / pnpm-lock.yaml)	State snapshot locking the exact dependency graph.	Exact versions installed, integrity hashes (SHA-512), resolved URLs, transitive dependencies.	Always commit to Git.

• dependencies vs. devDependencies:
  • dependencies: Packages required for the application to function in production (e.g., express, pg, zod). Installed when running npm install --omit=dev.
  • devDependencies: Tools used solely for local development, compilation, linting, or testing (e.g., typescript, jest, eslint, nodemon).
• Deterministic Builds (npm ci): In CI/CD pipelines and production deployments, run npm ci instead of npm install. It deletes node_modules, strictly validates against package-lock.json, bypasses version resolution, and throws an error if the lockfile and package.json are out of sync.

Semantic Versioning (SemVer)

SemVer follows the format MAJOR.MINOR.PATCH:

• MAJOR (X.0.0): Incompatible API changes (breaking changes).
• MINOR (0.X.0): Backwards-compatible new features.
• PATCH (0.0.X): Backwards-compatible bug fixes.

Package managers use prefix operators to manage automatic update ranges during npm install or npm update:

JSON

{
  "dependencies": {
    "exact-pkg": "1.4.2",      // Strict match: installs exactly 1.4.2
    "patch-pkg": "~1.4.2",      // Tilde (~): Accepts PATCH updates (>= 1.4.2 < 1.5.0)
    "minor-pkg": "^1.4.2",      // Caret (^): Accepts MINOR & PATCH updates (>= 1.4.2 < 2.0.0)
    "pre-zero-pkg": "^0.2.3",   // Exception: If MAJOR is 0, ^ locks MINOR (>= 0.2.3 < 0.3.0)
    "any-pkg": "*"              // Wildcard: Installs latest version (risks breaking changes)
  }
}

Local vs. Global Dependencies

• Local Dependencies (npm install <pkg>):
  • Stored in ./node_modules at the project root.
  • Scoped directly to the project, preventing version clashes between different projects on the same machine.
• Global Dependencies (npm install -g <pkg>):
  • Stored in the system’s global directory (e.g., /usr/local/lib/node_modules on Linux/macOS or %AppData%\npm on Windows).
  • Exposes binaries globally across your shell (e.g., npm, pnpm, pm2).
  • Avoid installing project-specific CLI tools globally (e.g., use local typescript or eslint instead of global installations to prevent environment mismatches across team members).

Modern Alternatives: PNPM & Yarn

• NPM: Standard default, uses a flattened node_modules layout. Can lead to “phantom dependencies” (importing packages your dependencies use that you didn’t explicitly define).
• Yarn: Introduced lockfiles, parallel downloads, and offline caching. Modern Yarn (Berry) supports Plug’n’Play (PnP), removing node_modules entirely in favor of an optimized lookup table.
• PNPM: Uses a content-addressable store on disk. All files are saved once in a global store and hard-linked into projects.
  • Saves gigabytes of disk space across multiple projects.
  • Uses isolated (non-flat) symlinked node_modules to eliminate phantom dependencies.

Scripts & Workflows

The scripts object in package.json allows defining custom commands, pre/post hooks, and environment configurations.

JSON

{
  "scripts": {
    "prebuild": "npm run clean",
    "build": "tsc",
    "postbuild": "node scripts/copy-assets.js",
    "start": "node dist/index.js",
    "dev": "node --watch --env-file=.env src/index.js",
    "test": "jest",
    "lint": "eslint .",
    "clean": "rm -rf dist"
  }
}

Lifecycle & Pre/Post Hooks

• Any script prefixed with pre or post executes automatically before or after the base command. Running npm run build sequentially executes:
  1. npm run prebuild
  2. npm run build
  3. npm run postbuild
• Default commands like npm start and npm test do not require the run keyword; custom scripts require npm run <name>.

Passing Arguments

To pass flags or options through an npm script to the underlying binary, separate them with a double hyphen (--):

Bash

# Passes --coverage down directly to Jest
npm test -- --coverage

# Passes a port flag to a local dev server
npm run dev -- --port=8080

NPX (Node Package Execute)

npx comes bundled with NPM and serves two primary purposes:

1. Running locally installed binaries without global installation: Instead of referencing ./node_modules/.bin/eslint ., run:Bashnpx eslint .
2. One-off execution of uninstalled packages:npx downloads the package to a temporary cache, executes the binary, and removes it immediately:Bashnpx create-next-app@latest my-app

Managing Secrets

Backend applications require sensitive credentials (database passwords, API tokens, encryption keys) that must never be hardcoded into the source code.

The .env File Conventions

1. .env: Contains real local development credentials. Must be added to .gitignore.
2. .env.example: A committed template displaying all required environment keys with blank or dummy values:Ini, TOML# .env.example PORT= DATABASE_URL= JWT_SECRET= NODE_ENV=

Method 1: Using dotenv (Standard Cross-Version Approach)

Install the package:

Bash

npm install dotenv

Load it as early as possible in your application lifecycle:

JavaScript

// ESM syntax
import 'dotenv/config';

// Or programmatic CJS syntax
// require('dotenv').config({ path: './custom.env' });

const port = process.env.PORT || 3000;
const dbUrl = process.env.DATABASE_URL;

if (!dbUrl) {
  throw new Error('FATAL: DATABASE_URL is not set in environment.');
}

Preloading without modifying source code (recommended for production builds):

Bash

node -r dotenv/config dist/server.js

Method 2: Native Node.js .env Loading

Node.js provides built-in .env file parsing via the --env-file CLI flag, eliminating the need for third-party packages like dotenv:

Bash

# Load default .env
node --env-file=.env server.js

# Load environment-specific file
node --env-file=.env.production server.js

Validating Environment Variables (Fail-Fast Pattern)

Relying on raw process.env leaves room for runtime bugs if a required string is missing or misspelled. Validating the environment at startup ensures the process fails fast before accepting traffic.

Example with Zod:

JavaScript

import { z } from 'zod';

const envSchema = z.object({
  NODE_ENV: z.enum(['development', 'test', 'production']).default('development'),
  PORT: z.coerce.number().default(3000),
  DATABASE_URL: z.string().url(),
  JWT_SECRET: z.string().min(32, 'JWT_SECRET must be at least 32 characters')
});

// Parse process.env; throws a detailed ZodError on failure
export const env = envSchema.parse(process.env);