Skip to content

Synchronization and Recovery

Error Handling and Retries

To safeguard data integrity during connectivity or document issues, the system follows a strict retry protocol before halting operations.

If a document fails to process (e.g., due to a database timeout or data conflict), the system executes the following:

  • The system makes three total attempts to process the document, with a 1-minute pause between each attempt.
  • If all three attempts fail, the document status is set to failed.
  • To prevent a backlog of errors, BS2SQL will automatically suspend processing for the entire environment.

Suspended State

A suspended state acts as a safety "circuit breaker," pausing active transfers while allowing new data to accumulate safely.

When the environment is suspended:

  • Document changes (inserts, updates, and deletes) are still captured and added to the queue.
  • The system stops attempting to push data to the SQL database until the underlying issue is addressed.

Monitoring and Recovery

Administrators can diagnose system health and restore synchronization directly through the BS2SQL setup page.

Diagnosing Issues

The setup page provides real-time visibility into the synchronization status:

  • The current operational state is displayed in the Status.
  • If the status is suspended, a Suspended Reason is provided to help identify the root cause, such as specific document errors or connectivity loss.

Resuming Operations

Once the underlying issue is resolved, follow these steps to restore the service:

  • Review the failed document(s) or connection settings.
  • Navigate to the BS2SQL setup page.
  • Click Resume to clear the suspended flag and restart the queue. The system will begin processing the next available entry that is not marked as failed.

Reconciliation

Reconciliation is a periodic verification process that ensures complete consistency between the BizzStream and SQL databases by comparing their entire datasets.

While the queue-based synchronization handles real-time changes, reconciliation acts as a safety net to detect and repair any discrepancies that may have occurred due to missed events, system failures, or data corruption.

How Reconciliation Works

The reconciliation process operates in two phases:

  1. All document definitions that are exposed to SQL are compared between the BizzStream and SQL databases to identify any structural differences.

  2. For each document definition, the actual documents in the BizzStream database are compared against their corresponding rows in SQL tables to detect missing, outdated, or orphaned records.

When reconciling each document definition and its documents, the system:

  • Compares documents/rows using their hash values to detect changes
  • Identifies records that need to be inserted, updated, or deleted
  • Applies the necessary changes to bring SQL into alignment with the BizzStream database

Locking Mechanism

During reconciliation, the system acquires an exclusive lock for the environment to prevent conflicts with normal queue processing. This ensures that reconciliation and real-time synchronization do not interfere with each other.

Triggering Reconciliation

Reconciliation runs automatically every day at 1 AM UTC. If the queue is actively processing documents when reconciliation is scheduled, the system will delay the reconciliation by 5 minutes and check again. This process repeats until the queue becomes available.

Administrators can also manually trigger reconciliation from the BS2SQL setup page by clicking the Reconcile button. This button is only enabled when:

  • The queue is not suspended
  • No documents are actively being processed