Class FileHelperFileSource<TOutput>

A dataflow worker that reads delimited (e.g. CSV) or fixed format data rows from one or more files, and sends them to a downstream worker.

Please see CSV Delimited and Fixed Format for how to specify the data format. Internally, this worker uses the http://www.filehelpers.net/ library to read the data, specifically the FileHelperAsyncEngine class.

Inheritance
Object
WorkerParent
WorkerBase
WorkerBase<FileHelperFileSource<TOutput>>
FileHelperFileSource<TOutput>
Implements
Inherited Members
Namespace: actionETL.FileHelper
Assembly: actionETL.dll
Syntax
[CLSCompliant(false)]
public class FileHelperFileSource<TOutput> : WorkerBase<FileHelperFileSource<TOutput>>, IDisposeOnFinished where TOutput : class
Type Parameters
Name Description
TOutput

A class that defines the format, columns, separators, column order etc. of the source data, as well as the type of the rows that will be passed to the downstream worker.

Constructors

FileHelperFileSource(WorkerParent, String, IEnumerable<String>)

Initializes a new instance of the FileHelperFileSource<TOutput> dataflow worker, that reads delimited (e.g. CSV) or fixed format data rows from one or more files, and outputs them to the downstream worker.

Declaration
public FileHelperFileSource(WorkerParent workerParent, string workerName, IEnumerable<string> fileNames)
Parameters
Type Name Description
WorkerParent workerParent

The parent worker or worker system that the new child worker will be added to. Cannot be null.
String workerName

Name of the worker.

Set to a prefix plus a trailing "/" (e.g. "MyPrefix-/") to generate a unique name from the prefix plus an increasing number starting at 1.

While less useful, set to null, whitespace or "/" to generate a unique name from the worker type plus an increasing number starting at 1.

The name cannot otherwise contain "/", and cannot start with double underscore "__".

IEnumerable<String> fileNames

The file names.
Exceptions
Type Condition
ArgumentException

workerName:

  • Workers with the same parent must have unique names.
  • Worker and worker system names cannot contain '/' or start with double underscore '__'.
ArgumentNullException

workerParent - All workers must have a parent. The top level workers have the worker system as parent.
InvalidOperationException
  • Cannot add child worker to parent which has completed. Are you adding it to the correct parent?
  • Cannot add worker to parent, since its children have been started. Are you adding it to the correct parent?

FileHelperFileSource(WorkerParent, String, Func<Boolean>, IEnumerable<String>)

Initializes a new instance of the FileHelperFileSource<TOutput> dataflow worker, that reads delimited (e.g. CSV) or fixed format data rows from one or more files, and outputs them to the downstream worker.

Declaration
public FileHelperFileSource(WorkerParent workerParent, string workerName, Func<bool> isStartableFunc, IEnumerable<string> fileNames)
Parameters
Type Name Description
WorkerParent workerParent

The parent worker or worker system that the new child worker will be added to. Cannot be null.
String workerName

Name of the worker.

Set to a prefix plus a trailing "/" (e.g. "MyPrefix-/") to generate a unique name from the prefix plus an increasing number starting at 1.

While less useful, set to null, whitespace or "/" to generate a unique name from the worker type plus an increasing number starting at 1.

The name cannot otherwise contain "/", and cannot start with double underscore "__".

Func<Boolean> isStartableFunc

Function to calculate the worker start constraint; it should return true for startable and false for not startable. Defaults to startable if null.
IEnumerable<String> fileNames

The file names.
Exceptions
Type Condition
ArgumentException

workerName:

  • Workers with the same parent must have unique names.
  • Worker and worker system names cannot contain '/' or start with double underscore '__'.
ArgumentNullException

workerParent - All workers must have a parent. The top level workers have the worker system as parent.
InvalidOperationException
  • Cannot add child worker to parent which has completed. Are you adding it to the correct parent?
  • Cannot add worker to parent, since its children have been started. Are you adding it to the correct parent?

FileHelperFileSource(WorkerParent, String, Func<Boolean>, String)

Initializes a new instance of the FileHelperFileSource<TOutput> dataflow worker, that reads delimited (e.g. CSV) or fixed format data rows from one or more files, and outputs them to the downstream worker.

Declaration
public FileHelperFileSource(WorkerParent workerParent, string workerName, Func<bool> isStartableFunc, string fileName)
Parameters
Type Name Description
WorkerParent workerParent

The parent worker or worker system that the new child worker will be added to. Cannot be null.
String workerName

Name of the worker.

Set to a prefix plus a trailing "/" (e.g. "MyPrefix-/") to generate a unique name from the prefix plus an increasing number starting at 1.

While less useful, set to null, whitespace or "/" to generate a unique name from the worker type plus an increasing number starting at 1.

The name cannot otherwise contain "/", and cannot start with double underscore "__".

Func<Boolean> isStartableFunc

Function to calculate the worker start constraint; it should return true for startable and false for not startable. Defaults to startable if null.
String fileName

The file name.
Exceptions
Type Condition
ArgumentException

workerName:

  • Workers with the same parent must have unique names.
  • Worker and worker system names cannot contain '/' or start with double underscore '__'.
ArgumentNullException

workerParent - All workers must have a parent. The top level workers have the worker system as parent.
InvalidOperationException
  • Cannot add child worker to parent which has completed. Are you adding it to the correct parent?
  • Cannot add worker to parent, since its children have been started. Are you adding it to the correct parent?

FileHelperFileSource(WorkerParent, String, String)

Initializes a new instance of the FileHelperFileSource<TOutput> dataflow worker, that reads delimited (e.g. CSV) or fixed format data rows from one or more files, and outputs them to the downstream worker.

Declaration
public FileHelperFileSource(WorkerParent workerParent, string workerName, string fileName)
Parameters
Type Name Description
WorkerParent workerParent

The parent worker or worker system that the new child worker will be added to. Cannot be null.
String workerName

Name of the worker.

Set to a prefix plus a trailing "/" (e.g. "MyPrefix-/") to generate a unique name from the prefix plus an increasing number starting at 1.

While less useful, set to null, whitespace or "/" to generate a unique name from the worker type plus an increasing number starting at 1.

The name cannot otherwise contain "/", and cannot start with double underscore "__".

String fileName

The file name.
Exceptions
Type Condition
ArgumentException

workerName:

  • Workers with the same parent must have unique names.
  • Worker and worker system names cannot contain '/' or start with double underscore '__'.
ArgumentNullException

workerParent - All workers must have a parent. The top level workers have the worker system as parent.
InvalidOperationException
  • Cannot add child worker to parent which has completed. Are you adding it to the correct parent?
  • Cannot add worker to parent, since its children have been started. Are you adding it to the correct parent?

Properties

Engine

Gets the underlying FileHelpers FileHelperAsyncEngine instance. Use this for customizing the data transfer, such as setting the text encoding and preprocess the data records. See CSV Delimited and Fixed Format for details.

Note: Retrieving and inspecting this property is thread-safe, but modifying the Engine state is not thread-safe, and should only be done from a single thread, before the worker has started.

Declaration
public FileHelperAsyncEngine<TOutput> Engine { get; }
Property Value
Type Description
FileHelpers.FileHelperAsyncEngine<TOutput>

The FileHelpers engine instance.
Remarks

Note that in rare cases, and only when using non-C# .NET languages, direct use of the Engine property can fail with a compile time error since the underlying FileHelpers library class (FileHelperAsyncEngine) is not CLS compliant (warning CS3003).

ErrorOutput

Gets the error output port for sending error rows to logging and an optional downstream worker.

Declaration
public ErrorOutputPort<StringRowErrors> ErrorOutput { get; }
Property Value
Type Description
ErrorOutputPort<StringRowErrors>

FileName

Sets the name of the file. Overrides any FileNames setting. Cannot be set after the worker has started running.

Note: This property is thread-safe.

Declaration
public string FileName { set; }
Property Value
Type Description
String
Exceptions
Type Condition
InvalidOperationException

Cannot set the value after the worker has started running.

FileNames

Sets the file names. Overrides any FileName setting. Cannot be set after the worker has started running.

Note: This property is thread-safe.

Declaration
public IEnumerable<string> FileNames { set; }
Property Value
Type Description
IEnumerable<String>

The file names.
Exceptions
Type Condition
InvalidOperationException

Cannot set the value after the worker has started running.

Output

Gets the output port for sending rows to the downstream worker.

Declaration
public OutputPort<TOutput> Output { get; }
Property Value
Type Description
OutputPort<TOutput>

Methods

RunAsync()

This method can be overridden to add custom functionality to the derived worker that runs before and after the row processing. In this case, the base class base.RunAsync() must be called for the worker to function correctly.

Typically, this worker is used without overriding this method.

Declaration
protected override Task<OutcomeStatus> RunAsync()
Returns
Type Description
Task<OutcomeStatus>

A Task describing the success or failure of the worker. An asynchronous async implementation would e.g. return OutcomeStatus.Succeeded on success, while a synchronous implementation would return OutcomeStatus.SucceededTask.
Overrides

Implements

See Also