> For the complete documentation index, see [llms.txt](https://docs.appstrategy.com/apprules-r-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.appstrategy.com/apprules-r-documentation/platform/platform-features/system-settings/data-sources/bulk-loading.md). # Bulk Loading ## Configuration Guide How to configure a target for high-volume loading with the Dynamics 365, Dynamics CRM, Salesforce, HubSpot and Zoho CRM connectors. **If you only read one thing:** leave the connection string alone and set **Loader Batch Size to 1000** on the target activity. That single change is most of the available speed-up. *** ### The two places settings live **The target activity** — *Target Loader & Logging Settings* dialog, on the target's property grid. Controls how many records appRules hands to the connector at a time. **The connection string** — *Bulk* and *Connection* categories in the connection editor. Controls how the connector sends them to the service. They work together, not against each other. Nothing in one overrides the other. *** ### Target activity settings | Setting | Recommended | What it does | | ----------------------- | -------------------- | -------------------------------------------------------------- | | **Load Option** | Bulk Load | Enables the bulk path. Without it, records save one at a time. | | **Linked Source** | your source activity | Optional. Leave empty if the source is a file. | | **Loader Batch Size** | **1000** | Records accumulated before being sent. | | **Job Submission Mode** | Sequential | See *Workers and threads* below. | | **Workers** | 1 Worker, 1 Thread | See *Workers and threads* below. | #### Loader Batch Size This is the setting that matters. It determines how many records are handed to the connector per flush, and therefore how many requests can be sent concurrently. | Loader Batch Size | Requests per flush | Sent concurrently? | | ----------------- | ------------------ | --------------------------- | | 100 | 1 | No — single request | | 500 | 5 | Yes | | **1000** | **10** | **Yes — recommended** | | 5000 | 50 | Yes, but see *Going faster* | At 100 the connector still batches, but there is only one request per flush, so there is nothing to overlap. From 500 upward the requests run in parallel. Higher is not automatically better. A larger batch means more records in flight when something fails, and more memory per flush. *** ### Connection string settings (Bulk category) **These settings are identical for Dynamics 365 and Dynamics CRM.** Salesforce, HubSpot and Zoho CRM have their own, smaller sets — see the connector sections at the end. **The defaults are correct for almost all loads.** Change them only if you have a specific reason. | Setting | Default | What it does | | ---------------------------------- | -------- | -------------------------------------------------------------- | | **Batch Chunk Size** | 100 | Records per request. | | **Batch Degree Of Parallelism** | 0 (auto) | Concurrent requests. 0 = decided automatically. | | **Max Auto Degree Of Parallelism** | 10 | Ceiling applied when the above is auto. | | **Max Batch Retries** | 4 | Retry attempts when the service asks you to slow down. | | **Max Connections Per Server** | 100 | Concurrent network connections. | | **Timeout** | **300** | Seconds before a request is abandoned. **Raise this from 60.** | #### Timeout — the one you should change The default of 60 seconds is too low for bulk loading. Under concurrency a single request can legitimately take longer than that, and when it times out the whole batch of records is reported as failed. **Set Timeout to 300.** #### Batch Degree Of Parallelism Leave at 0. The connector asks the service how much concurrency it allows and uses that, capped at *Max Auto Degree Of Parallelism*. You may see the service advertise a much higher number (48 is common). **Do not raise the cap to match it.** That figure is a ceiling on concurrent requests, not a sustainable rate — Dynamics also limits total processing time over a rolling window. Loads that use the full advertised concurrency typically succeed once and are then throttled for five minutes. If you do hit throttling, the connector halves its own concurrency automatically and carries on. You do not need to intervene. *** ### Recommended starting configuration **Target activity** ``` Load Option ............ Bulk Load Loader Batch Size ...... 1000 Job Submission Mode .... Sequential Workers ................ 1 Worker, 1 Thread ``` **Connection string** ``` Timeout=300; ``` Everything else in the Bulk category can be left at its default. *** ### Expected throughput Inserting `account` records, single worker, into a standard trial environment. **Dynamics 365** | Configuration | Time per 1,000 records | | --------------------------------- | ---------------------- | | Normal (non-bulk) | \~5 minutes 30 seconds | | Bulk Load, Loader Batch Size 100 | \~1 minute 50 seconds | | Bulk Load, Loader Batch Size 1000 | **\~26 seconds** | **Dynamics CRM** | Configuration | Time per 1,000 records | | --------------------------------- | ---------------------------------------- | | Normal (non-bulk) | \~4 minutes 30 seconds | | Bulk Load, Loader Batch Size 100 | \~2 minutes | | Bulk Load, Loader Batch Size 1000 | expected \~30 seconds (not yet measured) | **Salesforce** | Configuration | Time per 1,000 records | | -------------------------------- | ---------------------- | | Bulk Load, Loader Batch Size 100 | **\~5 seconds** | Salesforce is considerably faster than either Dynamics connector for the same volume — it accepts 200 records per call and processes them quickly. Loader Batch Size has far less influence here, because even 100 already fits in a single call. The two Dynamics connectors behave identically — they talk to the same service through the same mechanism — so the Dynamics CRM figure at Loader Batch Size 1000 should land close to the Dynamics 365 one. It is marked as expected rather than measured because that specific combination has not been timed. **HubSpot and Zoho CRM** Both send 100 records per call and neither runs requests concurrently, so expect throughput between the Dynamics and Salesforce figures — closer to Salesforce, since neither has Dataverse's per-record processing cost. Treat that as an expectation, not a promise, until a real load has been timed. Actual figures depend on your environment, the target table, and any plug-ins or workflows that run on record creation. Treat these as a guide, not a guarantee. A synchronous plug-in on the target table is the most common reason a load is slower than these numbers — no connector setting will compensate for it. *** ### Workers and threads The *Workers* setting adds more connections, each loading in parallel. It multiplies with the concurrency the connector already uses: two workers at 10 concurrent requests each means 20 requests in flight. **Two workers only take effect if Job Submission Mode is set to Parallel.** With Sequential, extra workers make no difference. Recommendation: **get the single-worker configuration working first.** Raise Loader Batch Size and confirm the throughput. Only add workers if that is not fast enough — and if you do, lower *Batch Degree Of Parallelism* so the total stays reasonable (roughly 20 concurrent requests across all workers). Each worker also opens its own connection and loads its own copy of the table metadata, which adds a few seconds of start-up per worker. *** ### Troubleshooting #### The load is slow Check *Loader Batch Size* first. At 100 there is no concurrency. Set it to 1000. #### Records fail with a length error Example: *the length of the 'address1\_county' attribute exceeded the maximum allowed length of '50'*. The source data contains a value longer than the target column allows. Trim or map the value in the workflow. The connector reports the real column lengths, so field mappings and validation rules in appRules will show the correct maximum. #### Some records failed but the error is unclear Check the connector's debug log. Set *Log Level* to 1 on the connection and look for: * `CHUNK FAILED` — a whole batch was lost; the message names the cause * `retry 1/4` — the service asked to slow down; the load continued * `WARNING: the service requires a ...s wait` — a throttling penalty. Not a hang; the load resumes on its own. #### The load appears to hang If the log shows a long wait warning, the service has applied a throttling penalty of up to five minutes. Wait for it. To avoid it recurring, lower *Loader Batch Size* or set *Batch Degree Of Parallelism* to a lower fixed value such as 5. #### Everything fails immediately Check the connection first with *Test Connection*. If that succeeds, check the log for the first error rather than the last — a single early failure often cascades. *** ### Salesforce differences Salesforce uses its own bulk mechanism and has fewer settings: | Setting | Default | Notes | | ------------------------------ | ------- | --------------------------------- | | **Max Collections Retries** | 4 | Retry attempts when rate-limited. | | **Max Connections Per Server** | 100 | Concurrent network connections. | Records are sent 200 at a time and requests are not run concurrently, so *Loader Batch Size* matters much less here than with Dynamics — a flush of 100 is already a single call. Anything from 100 to 1000 performs similarly; there is no need to tune it. Salesforce enforces a daily API call quota. If a load fails with a limit error, that quota is exhausted and retrying will not help until it resets. #### Bulk API V2 To use Bulk API V2, use the dedicated **Create Job and Upload Data** activity. It creates the job, uploads your CSV file, and closes the job, and gives you control over the object, action (insert/update/upsert/delete/query), external ID field, column delimiter and line ending. The job appears in Salesforce's Bulk Data Load Jobs monitor and the job Id and state are returned as activity outputs. *** ### HubSpot differences HubSpot has its own, smaller set of settings: | Setting | Default | Notes | | ------------------------------ | ------- | --------------------------------------- | | **Max Retries** | 4 | Retry attempts when rate-limited. | | **Max Connections Per Server** | 100 | Concurrent network connections. | | **Timeout** | 30 | **Raise this to 300 for bulk loading.** | Records are sent 100 at a time — HubSpot's Batch API limit — and requests are not run concurrently. **HubSpot's rate limits are the tightest of any of these connectors**, commonly 100 requests per 10 seconds. Batching helps rather than hurts here: 100 records in one request costs one call instead of a hundred. If you do hit the limit the connector waits for the interval HubSpot specifies and carries on, so a load slows down rather than failing. Note the Timeout default of 30 seconds is lower than the other connectors. With 100 records per request that can be tight, so set it to 300 before a large load. *** ### Zoho CRM differences | Setting | Default | Notes | | ------------------------------ | ------- | --------------------------------- | | **Max Rate Limit Retries** | 4 | Retry attempts when rate-limited. | | **Max Connections Per Server** | 100 | Not yet applied — see below. | | **Timeout** | 60 | Raise to 300 for bulk loading. | Records are sent 100 at a time — Zoho's per-call limit — and requests are not run concurrently. **Zoho meters API *****credits per call*****, not per record.** That makes batching unusually valuable: a row-at-a-time load spends the daily allowance a hundred times faster than it needs to. It also means a load that fails partway through has already consumed credits, so it is worth getting a small run right before a large one. *Max Connections Per Server* appears in the connection editor but is not yet applied to Zoho requests. It has no effect for now. *** ### A note on dates Values that arrive as text in a date column — which is what happens when the source is a CSV or other file — are converted to the format each service expects before being sent. A value such as `2026-08-24 13:25` is not valid for Salesforce and would otherwise be rejected with a JSON parser error. Two consequences worth knowing: * A value with no time component is sent as a plain date, because date-only fields reject a full timestamp. * A **text** field containing something that looks like a date will also be reformatted. If you have a genuine text field holding date-like content and it matters that it is preserved exactly, check the loaded values. Read-only fields are a separate matter. Some fields cannot be written at all — Salesforce's `LastActivityDate` is one — and mapping to them produces an error no setting will fix. Remove them from the field mapping. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.appstrategy.com/apprules-r-documentation/platform/platform-features/system-settings/data-sources/bulk-loading.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.