There may be situations when you need to transfer relatively big amounts of records (over 1000, or maybe as many as tens and hundreds of thousands) across your Quickbase apps and tables. This transfer must be guaranteed to be transactional in the sense that while you are doing the copying, the source data can be changing and the copy needs to take a consistent view of the source data and transfer that view to the destination table in a reliable and performant manner.
When you use Bulk Record Sets in the Quickbase channel you take advantage of a more efficient transfer of large numbers of records. When you batch the records you allow bulk operations to make fewer calls rather than one call per record.
Use Copy Records
In your Pipeline, add a Quickbase channel Bulk records Copy Records step. In this step, you can copy records from one Quickbase table to another using batch CSV export/import APIs.
Warning: This step cannot transfer non-ASCII characters unless UTF-8 is enabled at the Realm level. Enabling UTF-8 at the app level alone isn't enough to transfer data correctly.
When you select the Copy Record step:
Field details
Field | Description |
|---|---|
Quickbase account | Select the account you want to use for this copy transaction. |
Source table | Select the source table you want to export records from. |
Source table export fields | Select the fields you want to export into the CSV. |
Source table advanced query | Create a custom query to select the data you want to export into the CSV. For help with your query, see query operators. |
Destination table | Select the destination table where you want to import the records. |
Merge field | Select the field used to match records between the source and destination tables. Quickbase uses this field's value to decide whether to update an existing record or create a new one. Avoid using Record ID# for this when syncing two different tables—learn more about choosing a merge field. |
Terminate on error | Select whether or not you want to terminate execution on user error. No means the Pipeline will ignore user errors for this step and continue the execution and you will have control over the error handling. |
Field to map to column | Select the fields you want to map to each column. |
Choose a merge field
Warning: Don't use Record ID# as the merge field when syncing two different tables. Record ID# is generated independently in each table, so the same Record ID# value can point to two unrelated records in different tables. Using it as a merge key can overwrite or duplicate the wrong records.
Instead, create a dedicated field that stores a unique key you control, such as an external system ID, a formula combining fields that's guaranteed unique, or a GUID generated on record creation. Populate this field identically in both tables for records that represent the same real-world entity, then select it as the merge field.
Example
Say you're syncing Orders from Table A to Orders Archive in Table B on a schedule.
Avoid setting the merge field to Record ID#. This assumes Order #1042 in Table A always corresponds to Order #1042 in Table B, but Table B's Record ID# is its own independent counter—if Table B already has an unrelated record #1042, the sync updates the wrong row.
Instead:
Add a field called Order Unique Key to both tables (for example, a formula field combining Order Number and Customer ID, or a GUID set on record creation)
Make sure this field holds the same value in both tables for records representing the same order
Set Merge field to Order Unique Key
This way, the sync stays accurate even if record counts, deletions, or table structures diverge between the two tables over time.