Overview

SymmetricDS 3.14 release includes 20 features, 128 improvements, and 227 bug fixes.

Security Fixes

Issue Summary Severity

5351

Use PKCS12 for keystore by default

Medium

5376

Parameter to disable logging of application data from a batch error

Medium

5542

Apache commons-text version 1.9 security vulnerability

Medium

6034

Use the value of javax.net.ssl.trustStoreType as the truststore type if specified

Medium

6175

Upgrade Jetty

Medium

6282

Spring Framework URL Parsing with Host Validation

Medium

6298

Upgrade mysql, postgres libraries

Medium

6326

Spring Framework URL Parsing with Host Validation (part 2)

Medium

6626

Security review of springframework

Medium

Performance Fixes

Issue Summary Severity

5210

Routing reader may use multiple queries for a large number of gaps

Medium

6299

Hot spot removing old node sessions in mixed environment with versions < 3.11

Medium

What’s New

Operational Improvements

For large deployments, new endpoints can be scheduled to register automatically. Registration works across push links, in addition to pull links. Initial loads can now be monitored at the endpoint with the same level of detail that was previously only available at the source. Creation of tables now includes fractional second precision of time and timestamp columns. Routing query performance was improved when a large number of gaps is present. SQL-Server and Sybase trigger procedures were optimized for the case of a primary key changing, and a parameter of trigger.use.insert.delete.for.primary.key.changes was added to revert to the old behavior.

Issues

New Features

3.14.0
5178 - Push registration to nodes when group links indicate push
5180 - Schedule registration of a node for a window of time

3.14.1
5389 - Implement conflict_win_count and conflict_lose_count columns in sym_outgoing_batch and sym_incoming_batch

3.14.2
5427 - Creating a Parameter That, Upon Registration, Will Automatically Create A Group Link Between Nodes if None Exist

3.14.3
3313 - Propagate default GUID from MSSQL to SQLITE
5524 - Adding New Monitor Types For OS Load Average and OS Open File Handles
5551 - Add parameters that specify how to sync default values
5570 - Add engine name and server name variables to notification templates

3.14.4
5652 - Improve SQL Server DDL triggers and add DDL triggers for Oracle and Postgres

3.14.7
5757 - Add server parameter for setting a server’s cookie name
5774 - Support for capturing the row identifier on various platforms if available (default is false)
5775 - File sync batches to provide file name as the summary in outgoing and incoming batch.
5800 - Add a gradle task that generates javadoc and publishes it to the website

3.14.8
5863 - Add option to send additional batch error notifications when the number of batches in error increases
5884 - Add job monitor
5885 - MSSQL support for hierarchyid data types
5916 - New parameter to allow automatic loads to be sent based on configuration rather than just to and from registration server

3.14.9
1180 - Add option to symadmin --send-sql to specify a file containing the sql to send.

3.14.10
6013 - Add :SOURCE_NODE_ID, :SOURCE_EXTERNAL_ID, and :SOURCE_NODE_GROUP_ID variables to column match routers

3.14.11
6153 - Add parameters that can disable specific types of conflict resolution

Improvements

3.14.0
2972 - Registration order of nodes should not matter
3941 - Data gap route reader could use multiple queries when exceeding max gaps
4723 - Send column with Oracle DATE type to PostgreSQL
4724 - Send column with timestamp datatype from PostgreSQL to PostgreSQL
5138 - Build a cache manager to centralize the handling of data caches
5157 - Sybase and SQL Server update trigger should be made more efficient generating sym_data records when primary keys are changed
5198 - Clean up the consistent use of table constants for registration, trigger installation, extract, and export
5203 - Sync incoming load details to target node (table_reload_status, extract_request)
5210 - Routing reader may use multiple queries for a large number of gaps
5238 - Bulk loaders should report missing table
5265 - Update drivers, modules, and libraries
5269 - Variables of source catalog/schema names for use in target catalog/schema fields in routing
5278 - Alphabetize list of tables in user guide
5280 - Table DDL with timestamp and fractional second precision
5309 - Prepare for Gradle upgrade to version 8
5351 - Use PKCS12 for keystore by default
5353 - Add gpg code signing to jar files
5364 - Always execute shutdown hooks, even for embedded and war deployments
5376 - Parameter to disable logging of application data from a batch error

3.14.1
140 - Check to see that the SymmetricDS tables are created to use the Innodb storage engine on MySQL.
5415 - Detection of MSSQL Azure Managed Instance
5416 - Allow users to disable spatial data type support functions

3.14.2
5431 - Install triggers on registration server before configuration is complete
5433 - Sync triggers when parameters change that affect triggers
5440 - Change parameter treat.binary.as.lob.enabled to default to false
5449 - Add SOURCE_NODE_ID_FROM_DATA to VariableColumnTransform to allow using the source_node_id from the data
5450 - Routing default batch algorithm protection against large transactions
5453 - Backup keystore before saving it
5459 - Adding Log Info For Queries for Lookup Table Routers
5464 - Parameter to use old conflict pk detect and fallback
5476 - Ability to disable sync triggers when sync on incoming is enabled
5481 - Cast TEXT types in MS SQL to VARCHAR for SymmetricDS tables
5483 - Cast TEXT to VARCHAR for all tables
5486 - Ignore missing tables on config channel
5487 - Support snapshot fixes
5492 - Table reload request with "where" keyword gets error
5496 - Cast varchar to nvarchar
765 - Create table DDL and dbexport / dbimport should support generated / calculated / computed / virtual columns

3.14.3
1960 - Service start script always waits for 5s
5515 - When installing service for systemd, make sure to specify absolute path to java executable when creating systemd control file
5519 - Updating KafkaDataWriter to work with Filters
5520 - Acknowledgement gets null SQL message
5526 - Default max batch to send as 10
5535 - Improve how default values are handled when syncing DDL cross-platform
5536 - Add support for Postgres' inet, cidr, macaddr and macaddr8 data types
5547 - Treat any default value that contains "()" as a function
5553 - Allow spaces in values within value map column transform expression
5558 - Create table without defaults if in error
5563 - Add functionality to symadmin export-sym-tables to export alters when upgrading
5583 - Service wrapper wait for "started" message instead of 5 second timeout
5604 - Check sync URL ends with engine name
5605 - Parameter that alters table case should be for cross platform only
5606 - CPU monitor top threads should ignore threads waiting on I/O

3.14.4
4424 - Log monitor fails to insert because of duplicate key when logs are shared across instances of a node
5610 - SqlAnywhere support for v. 12 and higher
5630 - When deleting FK child rows during conflict resolution and old_data is null, query for the parent row on the target
5631 - Full load request does not support a reload select statement
5648 - Add "open registration" call to REST service
5650 - Problems replicating DDL to another node with the same database type but different database version

3.14.5
5636 - Only cancel a full load in progress if the reload select is empty
5662 - Snapshot util too slow for large multi-tenant deployment
5663 - Add table reload request to REST service
5671 - Bring back java routers
5673 - Sync triggers fails with deadlock on SQL-Server
5675 - Bidirectional sync causes need for trigger to be re-built on target side if table does not exist

3.14.6
5720 - sym_node trigger should capture changes even if auto.sync.configuration is set to false
5728 - AbstractSymmetricEngine.isConfigured() should use the cache when querying for sym_node table
5742 - Table sorting does not need to include sym tables
5747 - Read max-size VARCHAR columns as LONGVARCHAR columns on H2

3.14.7
5752 - Add support for H2 2.x
5753 - Reopening registration should reset the failed login count
5765 - Add quotation marks around values in threads-stats.csv file in support snapshot
5779 - Improve docs on db.init.sql property
5780 - Slow routing when using $(targetExternalId) variable
5782 - Unique Index with Multiple Nulls not Properly Handled for Certain Databases
5783 - SymmetricDS seems to require Postgres 11, but the docs don’t reflect that
5790 - Failed to decrypt node password from lost secret key in keystore
5805 - Allow specification of database platform of symmetric when DbDxport execution with load only/extract only/log miner
5810 - Redshift Bulk Loader increased versioning and dependencies
5815 - DDL Capture Changes - Three enhancements to DDL trigger capture
5817 - Snapshot util not returning
5823 - Capture timings when running Sync Triggers
5831 - Sync triggers by $(targetExternalId) and call after registration

3.14.8
2260 - Also sync the stored procedures with the schema
3476 - When a node is unregistered cancel all jobs / threads that are actively doing work for that node
5565 - Adding Character Set to Snapshot
5812 - MathColumnTransform uses Evaluator that is not thread-safe
5836 - Specify jaxb api and implementation so that libraries will be available when running in Java 8 through 17
5847 - Try again when foreign key correction fails
5853 - sym_monitor_event trigger should capture changes when monitor.events.capture.enabled=true even if auto.sync.configuration=false
5869 - Improve documentation of sym_node_group_link.sync_config_enabled
5873 - Open Registration in Rest API doesn’t allow user to specify node if 2 nodes have the same external ID
5886 - Monitor enhancements for centralized support
5893 - DbCompare should write to standard error and not continue with an invalid date-time format
5907 - Postgres support for ddl table creations with a default function on a GUID from MSSQL
5909 - Some methods are synchronized unnecessarily in AbstractParameterService
5922 - Added debug logging to File Sync Trigger Tracker
706 - Make sleep time between registration retries configurable

3.14.9
1063 - Add export-config and import-config command line options to symadmin
5921 - Adding Uni* type support to Sybase ASE
5932 - Update docs on using environment variables in engines properties (modified summary)
5937 - SymmetricDS User Guide Incorrectly Referencing Table Column Name Which Dropped From 3.14 Release Already
5944 - Adding log info for subselect routers
5960 - Add support for NTEXT to SQLAnywhere dialect

3.14.10
5966 - SQLAnywhere support for stream row
5980 - MonitorService should use the dirty SQL template when selecting from sym_monitor_event
5983 - Improve accuracy of CPU monitor
6006 - Prevent node from starting up if the sync.url is invalid
6010 - Add exception message to logging when bulk loader fails and falls back to default loading
6034 - Use the value of javax.net.ssl.trustStoreType as the truststore type if specified
6055 - Multi-threaded routing by channel

3.14.11
6064 - Slow extract and load when using $(targetExternalId) variable
6080 - Improve name and documentation for DBExport’s --symmetric option
6082 - SQLite nodes do not insert into sym_context correctly when disabling sync triggers
6095 - Contains big LOB check can be fooled by commas in data
6124 - Add documentation for PostGIS data types for Postgres
6131 - Too many "Expected but did not receive an ack for batch"

3.14.12
6164 - During an upgrade, log DDL before executing it instead of afterwards
6173 - Could not find trigger history, causes error of had X columns but expected Y
6177 - Snapshot util timeout for taking too long on table definitions
6199 - Remove securityToken parameter from /outgoingBatchSummary REST API call
6209 - "Node failed to authenticate" in server logs after server restarts or client session expires

3.14.13
6273 - Startup logging for specific node securities that can’t decrypt

3.14.14
6331 - Allow specification of initial load select for initial load of filesync channel

3.14.15
6364 - Add logging when instance ID changes
6374 - File Sync exceptions thrown by bsh script should be retried for transient errors

3.14.16
6446 - Provide the ability to specify a no lock hint when reading DDL

3.14.18
6666 - Initial load request should cancel outstanding load requests

Bug Fixes

3.14.0
2180 - H2 timestamp format creates conflict against ORACLE timestamp format
2254 - sym_outgoing_batch and sym_incoming_batch should both use the same time for last_update_time and create_time
2261 - Schema from Postgres outputs TIMESTAMPTZ that is not compatible with other databases
3724 - Conflict detection with Timestamps between DB2 and Derby
3778 - Send table fails when sending H2 Timestamp to MySQL
3962 - Oracle’s TIMESTAMP with Time Zone doesn’t map to MariaDB
3963 - char(n) and nchar(n) where n>255 is valid in oracle but maps to invalid types in MariaDB
4298 - Sycing schemas from MSSQL with column type CHAR(400) to MySQL fails
4525 - Database name with special character like minus sign, fails to create trigger
5122 - Wrong conversion from mssql datetime(7) to mysql/mariadb
5239 - Conflict resolution doesn’t work if timestamp is in PK and different fractional second precision between databases
5243 - Change documentation on outgoing batch errors to set the status to 'IG' instead of 'OK' so target will get notified
5304 - Snapshot Fails to Find SingleStore Enum
5305 - SingleStore Snapshot: Failed to execute SQL Error
5306 - SingleStore Snapshot: 'Table 'sym_outgoing_batch' doesn’t exist'
5308 - Unable to Retrieve Database Time for Load-Only and Log-Based

3.14.1
1282 - MySQL outputs schema with DEFAULT '0000-00-00' that is not compatible
1421 - DbFill error arithmetic overflow error with money data type
1453 - sym tables created in mysql should use innodb engine
2821 - dbcompare attempts to insert null into "not null default …​" field
3419 - MS SQL → SQLite, don’t translate "NEXT VALUE FOR" default value
3960 - NVARCHAR2() maps incorrectly to VARCHAR() when mapping from Oracle to MariaDB
3961 - Oracle XMLTYPE incorrectly mapped to invalid SQL Type SQLXML in MariaDB
3964 - Oracle’s REAL datatype should map to DOUBLE in MariaDB, not FLOAT
3965 - Oracle’s LONG type is incorrectly mapped to MariaDB’s MEDIUMTEXT. Should be LONGTEXT
3966 - Oracle’s CLOB (and NCLOB) type incorrectly maps to MariaDB’s MEDIUMTEXT
3999 - Unable to translate default column for (new sequentialid()) (very easy fix)
4001 - Default value being passed as string for MS Sql Server
4090 - Firebird ddl/schema problem when creating tables and deferring constraints
4134 - DB Compare does not handle Oracle Timestamp with LOCAL time zone columns properly
4191 - dbexport: MSSQL geometry → PGSQL varchar(2147483647); better would be TEXT
4217 - Does not replicate index with upper function on PostgreSQL
4359 - The handling of ''(empty string) default value of varchar columns of dbexport and dbimport is not correct for mssql server
4567 - sync.table.prefix parameter ignores for sequence in PostgreSQL
4667 - dbcompare prints each table multiple times
5106 - The PostgreSQL database cannot be synchronized under the USE_CHANGED_DATA and manual resolution strategy
5385 - Snapshots do not contain log files when the files are in a non-default location specified in log4j2.xml
5388 - Initial load delete or truncate with table transform
5393 - MySqlDdlReader.getTriggers() throws SQLException when MySQL database name contains special character
5395 - H2 database (or load only database) gets alter timestamp(0) every time starting
5406 - Deleting a trigger doesn’t inactivate its sym_trigger_hist entry when auto.sync.triggers.after.config.change=true
5413 - Bulk loaded flag is not being read from database for incoming and outgoing services
5414 - JDBC Bulk Transactions Fallback While Using PostgreSQL
5417 - Fix NPE for conflict detection when table does not have a PK and all columns used including those with null values
5421 - Initial load won’t start if registration_enabled is 1
5425 - Tables not created on reverse initial load

3.14.2
1753 - Default values that are functions do not work while creating tables in MySQL
3251 - Mysql BIGINT UNSIGNED processed as signed and gives error on larger than 63 bits numbers.
3845 - Primary key constraint violation while replicating single table from MSSQL 2017 into PostgreSQL 9.6
4116 - Syncing from Time to Timestamp (or vice versa) in Postgres produced inaccurate values
4295 - Detect when the order of a primary key changes in order to regenerate it
4335 - Multiple transform_table rows per source table cause multiplication of rows in outgoing batch
4514 - DbCompare: The output script to fix issues on the target does not support null values.
5392 - Unable to write to sym_outgoing_batch when using Postgres
5430 - Android not implemented exception when sym_monitor syncs
5435 - NumberFormatException when querying for an integer greater than Long.MAX_VALUE in SQL Explorer
5436 - Sybase ASE issues when upgrading SymmetricDS from an older version to 3.14.*
5439 - Conflict resolver fails if there are no primary keys on the target and it is set to not use primary keys from source
5441 - Windows service won’t start if tmp dir has trailing slash
5460 - Cannot write to more than one Kafka instance in a single instance of SymmetricDS
5472 - When a load filter changes catalog or schema the DML statement is not recreated with the updated values
5477 - Some default values do not sync correctly from Oracle to Oracle
5495 - Batch error missing method sendMissingForeignKeyRowsForLoad
5497 - NPE when running DbFill on a table with a foreign key that references a table in another catalog or schema
5501 - Registration left pending when client is 3.12 or newer and server is 3.11 or older
5502 - Fix json deserialization issue with log summary object
5503 - New config tables are being routed to old nodes
5504 - Oracle DATE type should not have its size specified
5505 - Oracle TO_TIMESTAMP() function gets surrounded by quotes when imported as a default value
922 - Float used on table without primary causes update to not capture

3.14.3
3109 - File replication routing job fails with error message when channel is not a file sync channel - needs better error message
3545 - Mapped default values lead to always detecting a model change
5456 - ORA-24816 when sync varchar2(4000) and long in same table
5463 - Initial load of a table with a self referencing FK will not load if out of order
5507 - Postgres LOCALTIMESTAMP function gets surrounded by quotes when imported as a default value
5510 - Multiple active trigger history for same table
5511 - Error request path not supported for /server/config
5512 - Default values for Oracle RAW columns get surrounded by quotes when exported
5513 - SQL Server GETDATE() and GETUTCDATE() functions get surrounded by quotes when imported as default values
5514 - SQL Server sysname type should not have its size specified
5523 - Create trigger hist if routing can’t find it
5529 - Cannot set up SQL Server log-based replication without "ALTER ANY DATABASE" permission
5531 - Multi-homed clustered nodes stuck in loop of retry and resend of batches
5542 - Apache commons-text version 1.9 security vulnerability
5544 - Initial load error on MySQL with parameter db.treat.date.time.as.varchar.enabled=true
5554 - Example 25 in User Guide is missing permissions
5557 - SQL-Server capture rows that exceed 4000 characters
5560 - ORA-06502: PL/SQL: numeric or value error: character string buffer too small
5571 - Unique index on function causes error when resolving a conflict
5580 - Script error from newer wins conflict resolution with old nodes
5582 - Database Platforms That Don’t Support SQL Queries Throw Error with SQL Explorer
5586 - Trigger creation fails when 2 new triggers have trigger IDs that are identical when shortened
5596 - Failure to Flush when using ConvertToReload router.
5603 - Parse exception of batch should remove it from staging

3.14.4
5597 - SymDS is not syncing specific record during initial load
5611 - Check sync URL ends with engine name or "sync"
5619 - Insert statements generated by DBExport contain question marks when there are null values
5624 - Deletes get ignored when foreign key children exist but cannot be found
5625 - Stack overflow while extracting batch containing a table removed from replication
5635 - Single Store extends MySQL dialect but does not need to check for an "innodb" engine
5638 - Flush cache of routers when a router is changed
5640 - Filtering transactions for snapshot gets stack overflow error and never finishes creating snapshot
5645 - Registration redirect URL does not have query parameters for older versions of SymmetricDS clients
5646 - DBExport incorrectly formats time data
5655 - sym_node_host trigger should capture changes even if auto.sync.configuration is set to false

3.14.5
5593 - Incorrect Logic in various DDLBuilders for Processing Changes
5598 - Sql Anywhere Drop Trigger
5664 - Tables are altered incorrectly on some platforms when their columns need to be made required or non-required
5679 - Extract fails with durationMillis must not be negative
5686 - MariaDB JDBC Driver version 3 does not support a negative fetch size to indicate the use of streaming
5692 - Pulling corrupted batch gets null pointer in loop
5693 - Fix multiple active trigger histories before sync triggers is run

3.14.6
3952 - Order table creation by foreign key dependency
5657 - SQLAnywhere CHAR is really a VARCHAR and should be mapped to VARCHAR for other dialects
5698 - Data truncation error when inserting/updating log event in sym_monitor_event on Interbase
5699 - Missing image in Manage Nodes Load Data Where Clauses
5700 - Grammar error (double word) in Manage Logging documentation
5702 - Missing image in Manage Incoming Loads
5708 - Sync table struct to target node report an error
5711 - When connecting to Azure, set platform version to SQL Server 2016
5718 - SQL Server DDL Builder needs to handle objects in different databases
5721 - Query tries to address mixed-case column without quotes (Postgres)
5722 - Tries to create foreign key before creating the referenced table
5723 - The schema of the table that is being created is used instead of correct one when creating a foreign key
5724 - SQL Server timestamp (rowversion) data type can not specify column size
5729 - DBImport- Allow the UI specification of the catalog and schema to override the catalog and schema specs in XML imported file
5731 - Functional indexes are only supported by Oracle
5737 - Server Time Offsets Causing an Error in Determining if a Node is Offline
5746 - When upgrading from a varchar(xxx) to longvarchar the change isn’t detected on H2

3.14.7
4310 - Replicating to postgres to postgres timestamp with default
5706 - "Data type unknown" error when updating sym_extract_request on Interbase
5766 - NPE when inserting into sym_incoming_batch on some platforms
5781 - DBCompare --use-sym-config true
5791 - Allow registration of other nodes after registration and initial load is complete
5799 - 3 tier deployment, 3rd tier does not get top level tier sym_node* tables
5802 - SQL Server and Sybase stream row on updates trigger syntax issue
5803 - Interbase fixes: failing to have prepared statement indicators in set and select portion of SQL statements
5807 - Sybase ASE gets Misinterpreted as Sybase SQL Anywhere
5809 - NullPointerException when writing a delete to Kafka when old data has not been captured
5811 - When Capture of Old Data is disabled, Deletes get an Index Out Of Bounds Error with Kafka
5822 - SQL Server create or alter trigger fails when table is not in default schema (dbo)
5826 - Data can become misaligned with column names during extraction after a column gets renamed
5828 - Class Not Found Exception when connecting to Redshift

3.14.8
4151 - MSSQL DDL ERROR: View / function / stored procedure
4952 - Cannot resolve monitors of type log
5314 - Infinite synchronization loop when 3 nodes are connected to each other and sync_on_incoming_batch = 1
5651 - Row in sym_table_reload_status can have incorrect values for completed, data_batch_loaded and finalize_batch_loaded
5834 - Unnamed constraint causes alter table error
5848 - Security Vulnerbility in Spring Framework
5865 - Estimated count from initial load does not work with SqlServer when using different schema than dbo
5867 - Prevent file sync from syncing all files when first scanning a directory when initial load is enabled
5880 - Transform documentation missing Java Column Transform Type
5892 - DbCompare fails to write comparison SQL for timestamp with timezone values
5894 - NotificationTypeEmail’s deserializeOfflineNodes method always returns empty collection
5897 - ValidatorException while validating self-signed X509 certificate
5901 - DBCompare produces output SQL with invalid syntax
5903 - Failing to create trigger on table
5904 - LOB data does not sync when sending a load with use_stream_lobs=1
5906 - Non transactional initial load setup can cause load events to be missed ini table_reload_status
5913 - Update postgres documentation on how to give access to schemas for a SymmetricDS user
5915 - Cannot Install swaggerui module by using symadmin ModuleManager

3.14.9
5931 - log.slow.sql.threshold.millis and log.sql.parameters.inline parameters do not work and are not documented
5934 - Getting null pointer from Column.java in new method anyPlatformColumnTypeContains on platformColumns
5935 - Failed to execute IHeartbeatListener PushHeartbeatListener java.lang.NullPointerException When Starting up SymmetricDS
5942 - SQLAnywhere DDL problems with dropping indexes and triggers, and trigger syntax issue for version 11 and earlier
5948 - KafkaDataWriter by ROW method has a trailing comma
5955 - Sybase Anywhere - channel_id size Problem on auto-create triggers
5961 - SQL Anywhere long varbit values are inconsistent between change data capture and initial load

3.14.10
5788 - The source_table_name wildcard behavior does not match the documentation on the website
5965 - Conflict Detection Fixes for Unique Keys
5971 - Loads fail when they include a table with a SQL Server uniqueidentifier column
5976 - User is incorrectly warned that table is missing primary key
5984 - Fixed compatibility for other databases to insert/update into Sybase ASE Unitypes
5986 - Method that selects from sym_node_security is not synchronized
5992 - Conflict resolver does not handle unique index violations correctly when a unique index ignores null values
6011 - Do not attempt to "create or alter" triggers on SQL Server 2016 RTM
6021 - SQLException when updating a non-OK outgoing batch on some database platforms
6043 - ConcurrentConnectionManager expects nodeId-channel in the white list, but only the nodeId is put in the white list
6051 - Sybase ASE result set has already been closed during trigger exists check

3.14.11
6032 - Multiple queues can cause contention when trying to register a node.
6071 - sym_node gets updated unnecessarily when heartbeat.update.node.with.batch.status parameter is disabled
6073 - Sync triggers fails when checking for multiple active trigger histories if Sybase ASE page size is 2K
6076 - Protect against exceptions in CPU monitor
6098 - Many router types attempt to handle non-DML data event types when they do not support them
6112 - Cancel of load at target does not clean up everything at the source and still shows active at target
6127 - Dbfill of nchar/nvarchar and avoid timestamp/rowversion
6133 - DB compare was incorrectly hex-encoding Unitype columns
6143 - Postgres timestamp with time zone columns get their values formatted incorrectly when extracted for a load
6150 - Firebird sym_hex UDF can cause an access violation

3.14.12
6162 - Snapshot error from database time on Sybase ASE/ASA
6175 - Upgrade Jetty
6192 - Adjusting unique indexes to non-unique when columns can be null should be able to be turned off by parameter
6211 - Registration error: sym_parameter changes causing sync triggers need to be limited to DML changes
6223 - Imported configuration sometimes gets deleted from sym_data

3.14.13
6245 - Cancel Load needs to make sure processes threads shut down before updating outstanding load request records
6263 - Include source_node_id in the where clause when updating or deleting from sym_table_reload_status
6266 - Prevent columns from being created with a size of 0
6269 - Prevent OracleSymmetricDialect from underestimating the current value of a sequence
6274 - FileSyncDataRouter gets FileTriggerRouters from database instead of cache
6282 - Spring Framework URL Parsing with Host Validation
6298 - Upgrade mysql, postgres libraries
6299 - Hot spot removing old node sessions in mixed environment with versions < 3.11

3.14.14
6311 - Failed to decrypt password counter measure if node registration is already open
6323 - Routing file deletes causes null pointer exception when capturing statistics
6326 - Spring Framework URL Parsing with Host Validation (part 2)
6327 - Specifying a reload channel ID for the channel_id in sym_table_reload_request results in the load getting stuck
6334 - File sync batches get synchronized when there is an error during extraction
6341 - Index out of bounds routing change to sym_node_group_link

3.14.15
6360 - NullPointerException when sending a load with file sync enabled and without a valid sym_trigger_hist row for sym_file_snapshot
6367 - Auto resolve delete of blocking rows when binary type in primary key
6384 - Error on registration batch or import of config from newer 3.14 or 3.15

3.14.16
6393 - When building DDL for SQL Server, default values of 'true' or null for bit columns get changed to 0
6400 - Issue with file sync when successful, retry is tried even when bean shell script is successful
6441 - NullPointerException when searching for a matching trigger history and row_data or pk_data is null
6467 - Symadmin export-sym-tables subcommand doubles the table prefix when called without the --alters option
6477 - If auto resolve missing foreign key fails, try again
6483 - Could not find trigger history causes error of wrong number of columns data validation in wrong column
6484 - Do not issue foreign key correction for non-DML events
6486 - Wildcard trigger definitions do not remove existing triggers when adding a NOT table name to wild carded definition
6489 - CPU monitor should use Java routine if native command not available
6502 - Sync on incoming batch setting does not get applied if a group link exists with the same source and target group ID
6506 - Values can be matched with the wrong columns during extraction when using targetExternalId variable and table definitions differ
6530 - Conflict resolution needs to set source node ID when executing conflict resolution for a SQL Server log mining node
6555 - CPU monitor is inaccurate for systems with a Mac or Linux OS and more than 1 core

3.14.18
6626 - Security review of springframework
6661 - Logging of exceptions on data load can show values used in the wrong order when conflict resolution is in play
6665 - New "Initial Load" not cancelling previous initial loads

Tables

The following changes were made to the definition of configuration and runtime tables. Table changes are applied to the database automatically using data definition language (DDL) during startup.

New Columns

SYM_EXTRACT_REQUEST
Column Name Description

source_node_id

Unique identifier for the node that will be the source of the extract.

SYM_INCOMING_BATCH
Column Name Description

bulk_loader_flag

A flag that indicates that this batch did or did not use the bulk loader.

SYM_JOB
Column Name Description

is_clustered

Whether to acquire a cluster lock or not.

SYM_NODE
Column Name Description

batch_last_successful

The last incoming or outgoing batch that was marked OK that is not on the config, heartbeat, or monitor channels.

data_rows_to_send_count

The number of of rows to replicated in batches. This field is updated as part of the heartbeat job if the heartbeat.update.node.with.batch.status property is set to true.

data_rows_loaded_count

The number of rows that has been successfully replicated since last purge. This field is updated as part of the heartbeat job if the heartbeat.update.node.with.batch.status property is set to true.

oldest_load_time

The oldest recorded sync time since the last purge.

most_recent_active_table

The most active table to sync recently (time based on last 5 minutes and doubles until a match is found or until 80 min.))

purge_outgoing_last_run_ms

The number of milliseconds that the purge job took on the last run. This field is updated as part of the heartbeat job if the heartbeat.update.node.with.batch.status property is set to true.

purge_outgoing_last_finish

The time the purge outgoing last finished succesfully. This field is updated as part of the heartbeat job if the heartbeat.update.node.with.batch.status property is set to true.

purge_outgoing_average_ms

The average time the purge outgoing has been run. This field is updated as part of the heartbeat job if the heartbeat.update.node.with.batch.status property is set to true.

routing_last_run_ms

The time it took for the the last routing job to finish succesfully. This field is updated as part of the heartbeat job if the heartbeat.update.node.with.batch.status property is set to true.

routing_last_finish

The time the routing job last finished succesfully. This field is updated as part of the heartbeat job if the heartbeat.update.node.with.batch.status property is set to true.

routing_average_run_ms

The number of milliseconds that the routing job averages. This field is updated as part of the heartbeat job if the heartbeat.update.node.with.batch.status property is set to true.

sym_data_size

The current size of the sym_data table. This field is updated as part of the heartbeat job if the heartbeat.update.node.with.batch.status property is set to true.

SYM_NODE_HOST_JOB_STATS
Column Name Description

error_flag

A flag that indicates that this job was in error.

error_message

The error message that occurred.

SYM_NODE_SECURITY
Column Name Description

registration_not_before

Allow registration beginning at this timestamp.

registration_not_after

Allow registration until this timestamp.

SYM_OUTGOING_BATCH
Column Name Description

bulk_loader_flag

A flag that indicates that this batch did or did not use the bulk loader.

SYM_TABLE_RELOAD_STATUS
Column Name Description

error_batch_id

The batch ID that is in error when the error_flag is 1.

batch_bulk_load_count

The number of batches that were loaded with the bulk loader.

SYM_TRIGGER_HIST
Column Name Description

is_missing_pk

Indicates whether the source table is missing a primary key.

Modified Tables

SYM_EXTRACT_REQUEST
  • Added primary keys: source_node_id

  • Added index idx_er_ld_src_nd (load_id, source_node_id)

  • Added index idx_er_src_nd_st (source_node_id, status)

Parameters

The following changes were made to add new parameters, modify their default value, modify their description, or remove them from use.

New Parameters

auto.reload.use.config

If this is true, a reload will use configuration to determine which nodes to send a load to. Default will send to the load to the node it regiestered (Default: false)

auto.resolve.foreign.key.violation.delete

If this is true, when a batch receives a foreign key violation due to the parent row being updated or deleted, all existing child rows will be deleted. (Default: true)

auto.resolve.primary.key.violation

If this is true, when a batch receives a primary key violation during an update, the blocking row will be replaced with the updated row. (Default: true)

auto.resolve.unique.index.ignore.null.values

If this is true, when a batch receives a unique index violation, the blocking rows for each unique index will only be deleted if the unique index has a value that is not null. This only has an effect if the auto.resolve.unique.index.violation parameter is set to true. (Default: true)

auto.resolve.unique.index.violation

If this is true, when a batch receives a unique index violation, the blocking rows for each unique index will be deleted. (Default: true)

conflict.default.pk.with.fallback

When enabled, the default conflict detection uses the primary key and the resolution uses fallback, which was the original default in version 3.11 and older. This is a convenient way to go back to the old default without having to configure a conflict on each group link. (Default: false)

create.index.convert.unique.to.nonunique.when.columns.not.required

If set to true, then convert unique indexes to non-unique when one of the columns is defined as not required (defined with the NOT NULL constraint). (Default: true)

data.create_time.timezone

Specify the timezone for the create_time value on sym_data when changes are captured. Only implemented for Oracle, Tibero, and PostgreSQL currently. (Default: )

data.flush.jdbc.batch.size

Flush size for JDBC batch mode used by services to save configuration. (Default: 10000)

dataloader.create.table.without.defaults.on.error

If set to true, when a table creation fails on a database platform that is different than the source database, try to create the table without default values. (Default: true)

dataloader.ignore.sql.event.errors

Indicate that the data loader should ignore errors while loading a SQL event and the execution of the statement fails. (Default: false)

dataloader.log.sql.params.on.error

Indicate that the data loader should log SQL parameter values when a batch fails, which can be helpful for debugging. Since SQL parameters will contain application data, some sites may need to turn this off for policy compliance. (Default: true)

default.values.to.leave.unquoted

A list of default values in CSV format that should not be quoted when synchronized from another node. (Default: )

default.values.to.translate

A list of pairs of default values in CSV format that tells SymmetricDS to translate the first value to the second value when synchronized from another node. (Default: )

file.sync.retry.count

How many times the file sync bean shell script will be attempted at the target node. The bean shell script will be attempted more than once if the exception java.nio.file.FileSystemException is thrown during the execution of the script. (Default: 2)

file.sync.retry.delay.ms

How long to pause in milliseconds between the attempts of the bean shell script that is run at the target node when the execution of the script fails. (Default: 5000)

log.slow.sql.threshold.millis

Defines the number of milliseconds before logging that a query is slow. (Default: 20000)

log.sql.parameters.inline

Defines whether the logging of SQL statements include the values inline or not. (Default: true)

mssql.use.snapshot.isolation

Specifies if snapshot isolation should be automatically turned on. (Default: false)

mssql.use.varchar.for.lob.in.sync

Use varchar(max) or nvarchar(max) when a column in SymmetricDS tables is set to long or nlong. This is for example necessary when using a _UTF8 of _SC collation. (Default: false)

opensearch.load.aws.access.key

The AWS secret access key (aws_secret_access_key) to use as credentials for uploading to S3 (Default: )

opensearch.load.aws.secret.key

The AWS secret access key (aws_secret_access_key) to use as credentials for uploading to S3 (Default: )

registration.auto.create.group.link

When this is set to true a group link will be created by default between two groups even if the user does not explicitly set one up. (Default: true)

registration.max.time.between.retries

The maximum time (in seconds) between node registration retries. (Default: 30)

registration.push.config.allowed

When group link is configured to push to clients, allow registration to work over push. Useful when client cannot reach server, so server pushes registration to client. (Default: true)

routing.data.reader.use.multiple.queries

Routing reader may run multiple queries for data, with each query including the maximum number of data gaps as specified by the routing.max.gaps.to.qualify.in.sql parameter. This method attempts to use the table’s index for quick results and avoid wasting time on filtering rows that were already routed (as used by the greater.than.query method). (Default: true)

routing.lock.timeout.ms

The amount of time a single routing worker node_communication lock will timeout after. (Default: 7200000)

routing.max.batch.size.exceed.percent

The percentage of the channel’s max batch size that a batch can exceed when seeking a transaction boundary using the default batch algorithm. Use zero to indicate that the batch size can grow as large as needed to include the complete transaction. For example, a setting of 100 percent with a channel’s max batch size of 1000 allows a batch size of 2000 to be routed before it will be forced as complete. This setting protects from a large transaction that causes batch sizes that far exceed the channel’s max size and have trouble loading on the target database. (Default: 100)

routing.thread.per.server.count

The number of threads created that will be used to route channels concurrently on one server in the cluster. (Default: 5)

routing.use.channel.threads

When enabled, use a thread per channel for parallel routing. (Default: false)

snapshot.max.batches

Max number of batches to write to statistics listing for support snapshot. (Default: 10000)

snapshot.max.files

Max number of files to write in directory listing for support snapshot. (Default: 50000)

snapshot.max.node.channels

Max number of nodes and channels for batch statistics, after which it will group by node only. (Default: 5000)

snapshot.operation.timeout.ms

Max time for a snapshot operation to complete, such as gathering table definitions, before it will be interrupted so the snapshot completes in a reasonable amount of time. (Default: 30000)

spatial.data.types.enabled

Determines if spatial data type functions will be installed. By default they will be installed but can be set to false to not install the additional spatial functions if they are not needed. (Default: true)

sybase.ase.convert.unitypes.for.sync

Converts unitypes in Sybase ASE to the corresponding type in the target database. (Default: false)

sync.triggers.fix.duplicate.active.trigger.histories

Whether or not duplicate active trigger histories will be detected and fixed so that the most recent will end up being the only active trigger history. (Default: true)

sync.triggers.reg.svr.install.without.config

Whether or not sync triggers job will install triggers on a registration server before configuration is created. When true, triggers are installed as soon as possible, right after configuration tables are created. When false, triggers are installed when the configuration includes one or more group links where this node is the source. A default of true avoids race conditions associated with registering nodes and performing operations before configuration is complete and triggers are installed. (Default: true)

trigger.capture.ddl.check.trigger.hist

Disable this property to capture all DDL changes regardless of whether the associated table has a corresponding row in sym_trigger_hist. MS SQL-Server, Oracle and Postgres only. See: trigger.capture.ddl.changes (Default: true)

trigger.use.insert.delete.for.primary.key.changes

For Sybase and SQL Server, if a primary key is changed, this parameter determines if a delete followed by an insert is captured for the row change instead of an update. When set to true, an update will be created if no primary keys are changed or if only one row is changed when a primary key change occurs, otherwise a delete followed by an insert is created for each row updated when a primary key change occurs and more than one row is updated by the SQL statement. If set to false, it will always create an update. This parameter, when changed, requires a restart of the SymmetricDS instance, followed by a rebuild of the triggers. (Default: true)

Modified Parameters

treat.binary.as.lob.enabled

Whether binary fields should be treated as lobs (Old Default: true) (New Default: false)

mysql.bulk.load.max.rows.before.flush

{REMOVED}

oracle.bulk.load.line.terminator

{REMOVED}

cloud.bulk.load.max.rows.before.flush

{REMOVED}

tibero.bulk.load.tbloader.options

{REMOVED}

cloud.bulk.load.s3.secret.key

{REMOVED}

cloud.bulk.load.s3.access.key

{REMOVED}

cloud.bulk.load.azure.blob.container

{REMOVED}

cloud.bulk.load.max.bytes.before.flush

{REMOVED}

redshift.bulk.load.max.bytes.before.flush

{REMOVED}

tibero.bulk.load.line.terminator

{REMOVED}

oracle.bulk.load.sqlldr.infile.charset

{REMOVED}

cloud.bulk.load.s3.bucket

{REMOVED}

tibero.bulk.load.dname

{REMOVED}

mssql.bulk.load.unc.path

{REMOVED}

redshift.bulk.load.s3.endpoint

{REMOVED}

tibero.bulk.load.field.terminator

{REMOVED}

mysql.bulk.load.local

{REMOVED}

oracle.bulk.load.sqlldr.options

{REMOVED}

oracle.bulk.load.sqlldr.cmd

{REMOVED}

redshift.bulk.load.s3.bucket

{REMOVED}

cloud.bulk.load.azure.account.name

{REMOVED}

mssql.bulk.load.row.terminator

{REMOVED}

mysql.bulk.load.max.bytes.before.flush

{REMOVED}

cloud.bulk.load.s3.region

{REMOVED}

cloud.bulk.load.s3.endpoint

{REMOVED}

cloud.bulk.load.azure.account.key

{REMOVED}

redshift.bulk.load.max.rows.before.flush

{REMOVED}

mssql.bulk.load.field.terminator

{REMOVED}

redshift.append.to.copy.command

{REMOVED}

tibero.bulk.load.tbloader.cmd

{REMOVED}

mysql.bulk.load.replace

{REMOVED}

mssql.bulk.load.fire.triggers

{REMOVED}

redshift.bulk.load.s3.secret.key

{REMOVED}

oracle.bulk.load.field.terminator

{REMOVED}

mssql.bulk.load.max.rows.before.flush

{REMOVED}

redshift.bulk.load.s3.access.key

{REMOVED}

oracle.bulk.load.ezconnect

{REMOVED}

cloud.bulk.load.azure.sas.token

{REMOVED}