This article explains how loops, lookups, and queries work in Pipelines, with a focus on performance.
Loops
A loop repeats a sequence of actions until a specific condition is met.
In this example, a For each loop is used to send a Slack message for each record in a list:
Lookups and Queries
Lookups and queries both retrieve data from connected systems, but they differ in scope, performance, and best use.
Lookup
A lookup searches for an item based on specific criteria and returns one result. They can use any unique field.
Lookups are easier to set up and perform faster. When possible, use a lookup to improve performance.
Query
A query searches all records in a selected account and returns a list of results. Queries are more flexible than lookups because they can filter on any field and return multiple results.
Queries are better suited for bulk operations that process or transfer groups of records. Although they use more resources and often require a loop, Jinja, or Insert to Quickbase step to process the results that are returned, they are more efficient overall when you’re handling many records at once.
Treat a query like a report—limit it to only the fields and records you need. For best performance, run the query first, then loop through the results or pass them to a bulk operation step.
Bulk data processing
When you need to move data at a higher volume, use one of these bulk data processing methods:
If you need to process a large number of incoming changes in real time, use a bulk trigger.