See Semver.
MAJOR or API version is incremented forMINOR or UPDATE version is incremented forPATCH version is incremented for🐞 Backwards-compatible bug fixes
📦 Minor packaging changes
💔 Dropped official support for Node v14, v16, and v18. Minimum Node.js version is now v20.
✨ Added startup validation for procps availability: BatchCluster now throws ProcpsMissingError during construction if the required ps command (or tasklist on Windows) is not available. This provides clear, actionable error messages instead of cryptic runtime failures. Resolves #13 and #39.
📦 Significant internal refactoring to improve maintainability:
ProcessPoolManager, TaskQueueManager, ProcessHealthMonitor, StreamHandler, ProcessTerminator)any with unknown throughout the codebase💔 Dropped official support for Node v16, which is EOL.
💔 Several methods, including BatchCluster#pids() were changed from async to sync (as they were needlessly async).
📦 A number of timeout options can now be validly 0 to disable timeouts:
spawnTimeoutMillistaskTimeoutMillis📦 Added eslint @typescript-eslint/await-thenable rule and delinted.
📦 Updated development dependencies and rebuilt docs
🐞 pidExists now handles EPERM properly (previous implementation would
mischaracterize pids as being dead due to insufficient permissions)
📦 Updated development dependencies and rebuilt docs
💔/✨ pidExists and killPid are no longer async, as process management
is now performed via node:process.kill(), instead of forking ps or tasklist.
📦 Updated development dependencies and rebuilt docs
🐞 Fix support for zero value of maxProcAgeMillis
📦 Updated development dependencies and rebuilt docs
🐞 Fix unref is not a function
📦 Updated development dependencies and rebuilt docs
Rate measurement.✨ If healthCheckCommand is set and any task fails, that child process will
have a health check run before being put back into rotation.
📦 Updated development dependencies and rebuilt docs
🐞 BatchCluster#maybeSpawnProcs in prior versions could spawn too many
processes, especially if process startup was slow. Heuristics for when to
spawn new processes now take into account pending task length and processes
busy due to initial setup.
📦 BatchCluster.vacuumProcs returns a promise that is only fulfilled after
all reaped child processes have completed BatchProcess.#end.
📦 BatchProcess.whyNotHealthy can now return startError.
📦 childEnd is now emitted only after the child process exits
📦 BatchCluster.#onIdle is debounced during the same event loop
📦 Added startup and shutdown spec assertions
📦 Updated development dependencies and rebuilt docs
📦 Add Rate.msSinceLastEvent
📦 Adjusted streamFlushMillis to remove onTaskData errors in CI.
✨ Exported Rate. You might like it.
✨ When child processes emit to stdout or stderr with no current task,
prior versions would emit an internalError. These are now given their own
new noTaskData events. Consumers may want to bump up streamFlushMillis if
they see this in production.
🐞/📦 Increased defaults for streamFlushMillis, added tests to verify noTaskData events don't happen in CI.
📦 Normalized node imports
✨/📦 Set minDelayBetweenSpawnMillis = 0 to fork child processes as soon as
they are needed (rather than waiting between spawn calls)
✨/📦 Set maxReasonableProcessFailuresPerMinute = 0 to disable process start
error rate detection.
✨/📦 New fatalError event emitted when
maxReasonableProcessFailuresPerMinute is exceeded and the instance shuts
itself down.
📦 New simpler Rate implementation with better time decay handling
📦 Several jsdoc improvements, including exporting WhyNotHeathy and
WhyNotReady
🐞 Fixed issue
#15 by
restoring the call to #onIdleLater when tasks settle.
🐞 Fixed issue with setMaxProcs which resulted in all idle processes being
reaped
📦 The idle event was removed. You weren't using it, though, so I'm not
bumping major.
📦 Process shutdown is handled more gracefully with new thenOrTimeout
(rather than the prior Promise.race call which resulted in a dangling
timeout)
📦 Updated development dependencies and rebuilt docs
.end() and .closeChildProcesses() closes all child processes in parallelBatchProcess interface✨ Process state improvements
💔 Renamed event s/childExit/childEnd/
💔 childEnd and childStart events receive BatchProcess instances now
💔 Renamed healthy state s/dead/ended/
📦 Made BatchProcess.whyNotHealthy persistent
📦 Added several more WhyNotHealthy values
📦 Perf: filterInPlace and count use for loops rather than closures
📦 Added spec to verify .end rejects long-running pending tasks
📦 Updated development dependencies and rebuilt docs
🐞/📦 BatchProcess exposes a promise for the completion of the startup task,
which BatchCluster now uses to immediately run #onIdle and pop off any
pending work.
📦 Updated development dependencies and rebuild docs
taskResolved on startup tasks.💔 The BatchProcessObserver signature was deleted, as BatchClusterEmitter is
now typesafe. Consumers should not have used this signature directly, but in
case anyone did, I bumped the major version.
✨ Added BatchCluster.off to unregister event listeners provided to BatchCluster.on.
📦 Private fields and methods now use the # private
prefix
rather than the TypeScript private modifier.
📦 Minor tweaks (fixed several jsdoc errors, simplified some boolean logic, small reduction in promise chains, ...)
📦 Updated development dependencies and rebuild docs
📦 Added BatchCluster.procCount and BatchCluster.setMaxProcs, and new
BatchCluster.ChildEndCountType which includes a new tooMany value, which
is incremented when setMaxProcs is set to a smaller value.
📦 Updated development dependencies
🐞/📦 BatchProcess now end on spurious stderr/stdout, and reject tasks if ending.
📦 Relaxed default for streamFlushMillis to deflake CI
💔/📦 RegExp pass/fail strings are escaped (which could conceivably be a breaking change, hence the major version bump)
📦 Refactored stdout/stderr merging code and added more tests
📦 Added new "taskResolved" event
📦 Rebuild docs
📦 Updated development dependencies
BatchProcessOptions fields✨ Added on("healthCheckError", err, proc) event
🐞 Fixed process start lag (due to startup tasks not emitting an .onIdle)
🐞 Reworked when health checks were run, and add tests to validate failing health checks recycle children
📦 Rebuild docs
💔 Several fields were renamed to make things more consistent:
BatchCluster.pendingTasks was renamed to BatchCluster.pendingTaskCount.BatchCluster.pendingTasks method now matches BatchCluster.currentTasks, which both return Task[].BatchCluster.busyProcs was renamed to busyProcCount.BatchCluster.spawnedProcs was renamed to spawnedProcCount.✨ Added support for "health checks" that run periodically on child processes.
Both healthCheckCommand and healthCheckIntervalMillis must be set to
enable this feature.
✨ New pidCheckIntervalMillis to verify internal child process state is kept
in sync with the process table. Defaults to every 2 minutes. Will no-op if idle.
✨ New BatchCluster.childEndCounts to report why child processes were recycled (currently "dead" | "ending" | "closed" | "worn" | "idle" | "broken" | "old" | "timeout" )
📦 Cleaned up scheduling: the prior implementation generated a bunch of
Promises per idle period, which was rough on the GC. Use of Mutex is now
relegated to tests.
📦 tsconfig now emits ES2018 output and doesn't have downlevelIteration,
which reduces the size of the generated javascript, but requires contemporary
versions of Node.js.
📦 BatchClusterOptions doesn't mark fields as readonly anymore
📦 Task has a default type of any now.
BatchCluster.currentTasksBatchCluster.closeChildProcesses() (ends child processes but doesn't .end() the BatchCluster instance)main branchNo new features in v6: just a breaking change so we can fix an old name collision that caused linting errors.
💔 Prior versions name-collided on Logger: both as an interface and as a
pseudonamespace for logger factories. This made eslint grumpy, and if anyone
actually used this bare-bones logger, it could have caused confusion.
Logger now references only the interface.
The builder functions are now named Log.
📦 Updated development dependencies
Deferred.resolve now requires an argument (as per the new Promise spec).
As this is just a typing change (and Deferred is an internal
implementation), I'm not bumping the major version.BatchCluster can now be created with a Logger thunk.maxIdleMsPerProcess option: automatically shut down idle child
processes to reduce system resource consumption. Defaults to 0, which
disables this feature (and prevents me from having to increment the major
version!)BatchProcess's streams could cause an infinite loop on .end() when
stdout was destroyed.BatchProcess.ready now verifies the child process still existsError: onExit(exit) called end()BatchCluster.end() should return a Deferred<void>onStartError and onTaskError didn't get emitted.shutdown()stdin.write() with try/catch that rejects the current task and
closes the current process.stdin.end() with try/catch (as .writable isn't reliable)BatchProcess (which incurred GC
overhead even when disabled)BatchCluster.options. Note that the object is frozen at
construction.BatchProcess.end() didn't correctly implement gracefully (which
resulted in spurious end(): called while not idle errors), and allowed for
multiple calls to destroy and disconnect from the child process, which may or
may not have been ill-advised.BatchCluster.isIdle. Updated development dependencies. Deflaked CI by embiggeningBatchClusterOptions.cleanupChildProcs, in case you want to handle
process cleanup yourself.maxProcs is respected again. In prior builds, if tasks were enqueued all
at once, prior dispatch code would only spin 1 concurrent task at a time.BatchProcess.end would result in different promise
resolution targets: the second call to .end() would resolve before the
first. This was fixed.minDelayBetweenSpawnMillis was added, to help relieve undue system load on
startup. It defaults to 1.5 seconds and can be disabled by setting it to 0.Deferred's warn log messages..pass and .fail regex now support multiple line outputs per task.🐞 BatchProcessOptions.pass
and .fail had poorly specified and implemented failure semantics. Prior
implementations would capture a "failed" string, but not tell the task that
the service returned a failure status.
Task Parsers already accept stdout and stderr, and are the "final word" in resolving or rejecting Tasks.
v5.2.0 provides a boolean to Parser's callable indicating if the wrapped
service returned pass or fail, and the Parser may return a Promise now, as
well.
There's a new SimpleParser implementation you can use that fails if stderr
is non-blank or a stream matched the .fail pattern.
🐞 initial BatchProcess validation uses the new SimpleParser to verify the
initial versionCommand.
✨ child process pids are delivered to event listeners on spawn and close. See BatchClusterEmitter.
🐞 fix "Error: end() called when not idle" by debouncing stdout and stderr
readers. Note that this adds latency to every task. See
BatchProcessOptions's
streamFlushMillis option, which defaults to 10 milliseconds.
🐞 RegExp for pass and fail tokens handle newline edge cases now.
📦 re-added tslint and delinted code.
ChildProcessFactory supports thunks that return either a ChildProcess or
Promise<ChildProcess>rejectTaskOnStderr API, which was added in v4.1.0 and applied to all
tasks for a given BatchCluster instance, proved to be a poor decision, and
has been removed. The Parser API, which is task-specific, now receives
both stdin and stderr streams. Parsers then have the necessary context to
decide what to do on a per task or per task-type basis.taskData events with the data and the
current task (which may be undefined) as soon as the stream data is emitted.NoLogger.
Consumers may use the ConsoleLogger or another Logger implementation as
they see fit.stderr emissions:
BatchProcess.rejectTaskOnStderr is a per-task, per-error predicate which
allows for a given error to be handled without always rejecting the task. This
can be handy if the script you're wrapping (like ExifTool) writes non-fatal
warnings to stderr.BatchProcessOptions.pass and BatchProcessOptions.fail can be RegExp
instances now, if you have more exotic parsing needs.💔 Using Node 8+ to determine if a process is running with kill(pid, 0)
turns out to be unreliable (as it returns true even after the process exits).
I tried to pull in the best-maintained "process-exists" external dependency,
but that pulled in 15 more modules (this used to be a zero-deps module), and
it was extremely unperformant on Windows.
The TL;DR: is that running(pid) now returns a Promise<boolean>, which had
far-reaching signature changes to accomodate the new asynchronicity, hence the
major version bump.
💔 In an effort to reduce this library's complexity, I'm removing retry functionality. All parameters associated to retries are now gone.
✨ Internal state validation is now exposed by BatchCluster, and is used by tests to ensure no internal errors happen during integration tests. Previously these errors were simply logged.
Logger methods, withLevels, withTimestamps, and filterLevels
were shoved into a new Logger namespace.Error: prefixesonExit that aren't fatal), we now log .error
rather than throw Error() or ignore.Task promises are only rejected with Error instances now. Note
that also means that BatchProcessObserver types are more strict. It could be
argued that this isn't an API breaking change as it only makes rejection
values more strict, but people may need to change their error handling, so I'm
bumping the major version to highlight that. Resolves
#3. Thanks for the
issue, Nils Knappmeier!/PID option seemed to work downcased, but the docs say
to use uppercase, so I've updated it.(v2.1.2 is the same contents, but np had a crashbug during publish)
end for BatchProcess, which may prevent very long-lived
consumers from sporadically leaking child processes on Mac and linux.Logger.trace and moved logging related to per-task items down
to trace, as heavy load and large request or response payloads could
overwhelm loggers. If you really want to see on-the-wire requests and results,
enable trace in your debugger implementation. By default, the
ConsoleLogger omits log messages with this level.BatchClusterObserver with a simple EventEmitter API on
BatchCluster to be more idiomatic with node's APImaxTaskErrorsPerProcess parameter was removed)Rate is simpler and more accurate now.BatchClusterObserver for error and lifecycle monitoringtimers.setInterval. May address this
issue.
Thanks for the PR, Tim Fish!BatchProcess.end() to use until() rather than Promise.race,
and always use kill(pid, forced) after waiting the shutdown grace period
to prevent child process leaks.Logger.setLogger() for debug, info, warning, and errors. debug and
info defaults to Node's
debuglog,
warn and error default to console.warn and console.error,
respectively.tsc provides good lint coverage nowdelay now allows
unrefing the
timer, which, in certain circumstances, could prevent node processes from
exiting gracefully until their timeouts expiredkill() and running() from BatchProcesstaskkill on windows and kill -9
on other unix-like platforms if they don't terminate after sending the
exitCommand, closing stdin, and sending the proc a SIGTERM. Added a test
harness to exercise.mocha tests don't require the --exit hack anymore 🎉.running() works correctly for PIDs with different owners now.yarn upgrade --latestprettier and delintedconsole.log with a call to log.maxProcs wasn't always utilized by onIdle, which meant in
certain circumstances, only 1 child process would be servicing pending
requests. Added breaking tests and fixed impl.kill(0) calls to verify the child
processes are still running work across different node version and OSesBatchProcess (whose API should not be accessed
directly by consumers, so the major version remains at 1).end()BatchCluster to BatchProcessBatchCluster now has a force-shutdown exit handler to accompany the
graceful-shutdown beforeExit handler. For reference, from the
Node docs:The 'beforeExit' event is not emitted for conditions causing explicit termination, such as calling process.exit() or uncaught exceptions.
Rate's time decay in the interests of simplicityprocessFactory or versionCommand fails more often than a given
rate, BatchCluster will shut down and raise exceptions to subsequent
enqueueTask callers, rather than try forever to spin up processes that are
most likely misconfigured.maxProcAgeMillis, and restarted as neededBatchCluster now practices good listener hygene for process.beforeExit