Interface TraversalConfig

The configuration with which a traverser is created.

interface TraversalConfig {
    batchSize: number;
    maxBatchRetryCount: number;
    maxConcurrentBatchCount: number;
    maxDocCount: number;
    sleepTimeBetweenBatches: number;
    sleepTimeBetweenTrials: number | ((lastTrialIndex) => number);
}

Properties

batchSize: number

The number of documents that will be traversed in each batch.

Default Value

250
maxBatchRetryCount: number

The maximum number of times the traverser will retry processing a given batch, in case of an error. By default, batches are not retried.

Remarks

This field must be a non-negative integer representing the maximum number of times the traverser will retry processing a given batch. By default, the traverser invokes the batch callback only once i.e. with 0 retries but, if maxBatchRetryCount > 0, it will keep invoking the callback until it succeeds or the total number of retries reaches maxBatchRetryCount.

Default Value

0
maxConcurrentBatchCount: number

The maximum number of batches that can be held in memory and processed concurrently.

Remarks

This field must be a positive integer representing the maximum number of batches that you wish the traverser to process concurrently at any given time. The traverser will pause when the number of batches being processed concurrently reaches this number and continue when a batch has been processed. This means that the higher the value of maxConcurrentBatchCount, the more memory the traverser will consume but also the faster it will finish the traversal.

Example

const projectsColRef = firestore().collection('projects');
const traverser = createTraverser(projectsColRef, {
batchSize: 500,
maxConcurrentBatchCount: 20,
});

By providing this config we have indicated that we want each batch to contain 500 documents and the traverser to process 20 batches concurrently at any given time. This means we have ensured that our machine can handle 500 * 20 = 10,000 documents in memory at any given time.

Default Value

1
maxDocCount: number

The maximum number of documents that will be traversed.

Default Value

Infinity

sleepTimeBetweenBatches: number

The amount of time (in ms) to "sleep" before moving on to the next batch.

Default Value

0
sleepTimeBetweenTrials: number | ((lastTrialIndex) => number)

A non-negative integer or a function that takes the 0-based index of the last trial and returns a non-negative integer indicating the amount of time (in ms) to "sleep" before retrying processing the current batch. This is useful if you want to implement something like exponential backoff.

Type declaration

    • (lastTrialIndex): number
    • Parameters

      • lastTrialIndex: number

      Returns number

Default Value

1000

Generated using TypeDoc