Release Notes

This article lists the main changes in each actionETL release.

The library and the project template nuget packages are available from nuget.org/profiles/envobi.

To update the templates, simply re-install them:

dotnet new --install actionETL.templates.dotnet

To update a project, execute in the project folder:

dotnet add package actionETL

Version 0.40.0

This release adds an additional MySQL/MariaDB provider that supports high performance bulk insert, adds new sample projects, and also provides a number of smaller additions, changes and fixes.

Added new Samples projects:
- AggregateCsvCreateInsertTable
- CopyFileIfExists
- MultipleWorkerSystemsDataflow
- ReadSortWriteXlsx
Added additional MySql/MariaDB provider Adb MySqlConnector Provider supporting high performance Bulk Insert
Added transaction support to AdbMySqlConnectorBulkInsertTarget<TInput> and AdbSqlClientBulkInsertTarget<TInput>
Added AdbSqlClientBulkInsertTarget.ColumnOrderHints and renamed SqlBulkCopyOptions to CopyOptions
AdbInsertTarget<TInputError> related:
- Reduced AdbInsertTarget database round-trips
- Renamed SetMaxRows() to SetRowLimits(Int32, Int64)
- Deprecated SetSingleRow()
- Support setting default transaction size and disabling transactions
- Fixed column mapping bug
- ErrorOutput:
  - Fixed redirection of rows on insert exception
  - Linking disables creating transactions, and throws if there is a pre-existing transaction
Added AdbProvider.WithSql92InsertStatement() to simplify setting default batch and transaction sizes for AdbInsertTarget<TInputError>
IAdbInsertStatementService changes for AdbInsertTarget<TInputError>:
- Renamed DefaultParametersPerStatement to DefaultValuesPerBatch
- Renamed MaxParametersPerStatement to MaxValuesPerBatch
- Added AConfigSetting InsertStatementValuesPerBatch and InsertStatementValuesPerTransaction
- Added rowsPerBatch parameter to Create()
ErrorOutputPort<TError>:
- Added SendErrorBuffer() for sending a batch of error rows
- Only log exception and message on the first row in a batch
- Fixed bug when logging a null column
Removed deprecated InputPort.TakeRows()

Package dependencies:

FileHelpers 3.4.1 > 3.4.2
Microsoft.Data.SqlClient 2.0.1 > 2.1.2
MySqlConnector 1.3.3 added
MySql.Data 8.0.22 > 8.0.24
NodaTime 2.4.8 > 2.4.13
Npgsql 4.1.6 > 4.1.9
System.Net.NameResolution removed
System.Net.Primitives removed

Version 0.39.0

Added a free Community edition
Added .NET 5 support
API changes:
- Added ProgressStatusResult<TResult> and extended OutcomeStatusResult<TResult>
- Changed OnOutputRowDemand() to return ProgressStatusResult<TResult>
Added package dependency Microsoft.DotNet.PlatformAbstractions 3.1.6 and improved logging of .NET runtime version as well as operating system version
Bumped package dependency versions:
- Microsoft.Data.SqlClient 2.0.0 > 2.0.1
- MySql.Data 8.0.21 > 8.0.22
- NLog 4.7.3 > 4.7.5
- Npgsql 4.1.4 > 4.1.6
- System.CodeDom 4.7.0 > 5.0.0
- System.Collections.Immutable 1.7.1 > 5.0.0
- System.Configuration.ConfigurationManager 4.7.0 > 5.0.0
- System.Data.Odbc 4.7.0 > 5.0.0
- System.Data.SQLite.Core 1.0.113.1 > 1.0.113.6
- System.Runtime.CompilerServices.Unsafe 4.7.1 > 5.0.0

Version 0.38.0

This release brings enhanced database support across all database providers, for both on-premises and cloud deployments:

Improved GetTableColumnsAsync(String) and GetTableTypeAsync(String), which also helps AdbInsertTarget<TInputError> and AdbTableNonQueryWorker:
- MariaDB™ and MySql™:
  - Fixed quoting bug.
- MySql™:
  - Fixed table metadata bug for 4GB column types longblob and longtext.
- PostgreSQL®:
  - Added support for temporary tables and PostgreSQL schema search_path.
  - Added support for BitArray to bit varying type mapping (in addition to the existing varbit mapping).
  - Fixed bug in GetTableColumnsAsync(String) overloads when no schema specified.
- SQLite: Added support for views.
- SQL Server®: Improved Precision, Scale, LengthInBytes, and LengthInCharacters column metadata from GetTableColumnsAsync(String) overloads.
AdbExecuteScalarWorker<TResult> converts (with ChangeType(Object, Type)) the worker result if casting fails, thereby handling more data type mismatches.
Verified actionETL on Azure for MariaDB, MySql, PostgreSQL, and Azure SQL Database.

Version 0.37.0

actionETL.nupkg

Added commands to AdbTableNonQueryWorker for dropping views, and renamed table commands
Added AdbTableType for detecting table-like type (table, view, or table-valued function). Renamed TableExistsAsync() to GetTableTypeAsync(String)
Changed GetTableColumns() to asynchronous GetTableColumnsAsync(String) overloads.
Added StringNumberRow dataflow row class to simplify extracting text lines with line numbers
Fixed EnumerableSource<TOutput> dispose bug
Deprecated InputPort<T>.TakeRows() since some use cases could deadlock dataflow. Use TakeBuffer() instead.

actionETL.templates.dotnet.nupkg

Fixed the actionetl.console template getting the correct namespace

Version 0.36.1

Added RepeatRowsSource.SendTemplateRows property for sending template rows without cloning them.
Improved performance and fixed stack overflow for dataflows using >10,000 linked workers with the same parent.

Version 0.36.0

Moved the library package and templates package distribution to nuget.org
Added automatically making worker names unique when specified with trailing "/"
Bumped package dependency versions:
- MySql.Data 8.0.20 > 8.0.21
- NLog 4.7.2 > 4.7.3
- Npgsql 4.1.3.1 > 4.1.4
- Sentry.PlatformAbstractions 1.1.0 > 1.1.1
- System.Net.Primitives 4.3.0 > 4.3.1

Version 0.35.x

Added .NET Standard and .NET Core support
Added Linux support
Added ‘dotnet new’ project templates
Bumped package dependency versions:
- EPPlus 4.5.3.2 > 4.5.3.3
- Microsoft.Data.SqlClient 1.1.3 > 2.0.0
- MySql.Data 8.0.19 > 8.0.20
- NLog 4.6.8 > 4.7.0
- NodaTime 2.4.7 > 2.4.8
- Npgsql 4.1.2 > 4.1.3.1
- System.Collections.Immutable 1.6.0 > 1.7.0
- System.Data.SQLite.Core 1.0.112.2 > 1.0.113.1
- System.Runtime.CompilerServices.Unsafe 4.6.0 > 4.7.1
- System.Threading.Tasks.Extensions 4.5.3 > 4.5.4

Version 0.34.x

Added PostgreSQL database support

Version 0.33.x

Added SQLite database support