Class WorkerParent
A base class for WorkerSystem and WorkerBase with shared functionality for managing child workers, RunAsync(), etc.
Implements
Namespace: actionETL
Assembly: actionETL.dll
Syntax
public abstract class WorkerParent : IDisposeOnFinished
Properties
AggregateErrorOutputRows
Gets the total number of rows sent to any error output port in this or any descendant worker that has completed.
Note that this aggregates RowsReceived, i.e. all rows that were received by the error output ports, even if they were not necessarily sent to the downstream worker. Also see TotalRowsSent.
Note: This property is thread-safe.
Declaration
public long AggregateErrorOutputRows { get; }
Property Value
Type | Description |
---|---|
Int64 | The total number of error output port rows. Returns |
AggregateOutputRows
Gets the total number of rows sent to any (non-error) output port in this or any descendant worker that has completed.
Note: This property is thread-safe.
Declaration
public long AggregateOutputRows { get; }
Property Value
Type | Description |
---|---|
Int64 | The total number of (non-error) output port rows. Returns |
AggregateWorkersCompleted
Gets the total number of descendant workers plus this worker that ran and completed.
Note: This property is thread-safe.
Declaration
public long AggregateWorkersCompleted { get; }
Property Value
Type | Description |
---|---|
Int64 | The number of completed workers. Returns |
Children
Returns a list with all current worker children. It is often used in debugging for navigating the worker hierarchy, and in testing for checking worker status.
Note: This property is thread-safe. On each access, it returns a new list with a copy of all current children. In performance sensitive areas, avoid reading this property a large number of times (e.g. once for every dataflow row).
Declaration
public IReadOnlyList<WorkerBase> Children { get; }
Property Value
Type | Description |
---|---|
IReadOnlyList<WorkerBase> |
DebugCommands
Gets or sets commands for when to launch or break a debugger when debugging workers
and the worker system (i.e. WorkerParent instances).
Multiple commands can be set at the same time by combining them with bitwise OR
(|
in C#).
The various "On..." commands defines when to break or launch the debugger, while Launch and Break defines what to do, i.e. launching and breaking the debugger. In most debugging scenarios, both one or more "On..." commands, as well as either Launch or Break, should be set.
There are also several predefined combinations, e.g.
BreakOnCompleted that is very useful for viewing
WorkerParent
s after they complete.
Note: Any Launch command will be automatically
downgraded to a
Break command after the first attempt to launch a debugger
for this WorkerParent
.
Note: The WorkerParent
state is checked for a match both when this property is set,
and when the WorkerParent
state changes.
Note: This property is thread-safe.
Declaration
public DebugWorkerParentCommands DebugCommands { get; set; }
Property Value
Type | Description |
---|---|
DebugWorkerParentCommands |
See Also
HasChildren
True if the worker has any children.
Note: This property is thread-safe.
Declaration
public bool HasChildren { get; }
Property Value
Type | Description |
---|---|
Boolean |
InstantCompleted
Gets the nullable instant the worker completed execution.
Note: This property is thread-safe.
Declaration
public Instant? InstantCompleted { get; }
Property Value
Type | Description |
---|---|
Nullable<Instant> | The nullable instant.
Use |
InstantCreated
Gets the instant the worker or worker system was created (which for workers is also when it was added to its parent).
Note: This property is thread-safe.
Declaration
public Instant InstantCreated { get; }
Property Value
Type | Description |
---|---|
Instant |
InstantStarted
Gets the nullable instant the worker started execution.
Note: This property is thread-safe.
Declaration
public Instant? InstantStarted { get; }
Property Value
Type | Description |
---|---|
Nullable<Instant> | The nullable instant.
Use |
IsCanceled
Gets a value indicating whether this instance is canceled.
Note: This property is thread-safe.
Declaration
public bool IsCanceled { get; }
Property Value
Type | Description |
---|---|
Boolean |
|
IsCompleted
Gets a value indicating whether this instance is completed, i.e. has stopped running.
Note: This property is thread-safe.
Declaration
public bool IsCompleted { get; }
Property Value
Type | Description |
---|---|
Boolean |
|
IsCreated
Gets a value indicating whether this instance is created, but has not progressed to any later state.
Note: This property is thread-safe.
Declaration
public bool IsCreated { get; }
Property Value
Type | Description |
---|---|
Boolean |
|
IsError
Gets a value indicating whether this instance has errored.
Note: This property is thread-safe.
Declaration
public bool IsError { get; }
Property Value
Type | Description |
---|---|
Boolean |
|
IsFailed
Gets a value indicating whether this instance has completed unsuccessfully.
Note: This property is thread-safe.
Declaration
public bool IsFailed { get; }
Property Value
Type | Description |
---|---|
Boolean |
|
IsFatal
Gets a value indicating whether this instance has fatally faulted.
Note: This property is thread-safe.
Declaration
public bool IsFatal { get; }
Property Value
Type | Description |
---|---|
Boolean |
|
IsRunning
Gets a value indicating whether this instance is running.
Note: This property is thread-safe.
Declaration
public bool IsRunning { get; }
Property Value
Type | Description |
---|---|
Boolean |
|
IsSucceeded
Gets a value indicating whether this instance has succeeded.
Note: This property is thread-safe.
Declaration
public bool IsSucceeded { get; }
Property Value
Type | Description |
---|---|
Boolean |
|
Item[String]
Gets the child worker with the specified worker name.
There is a small overhead when retrieving the worker by name. Consider storing and using the worker instance in a variable in performance sensitive areas.
Note: This property is thread-safe.
Declaration
public WorkerBase this[string workerName] { get; }
Parameters
Type | Name | Description |
---|---|---|
String | workerName | Name of the child worker. |
Property Value
Type | Description |
---|---|
WorkerBase | The child worker. To access members from derived workers, the returned value must first be cast to the appropriate worker type. |
Exceptions
Type | Condition |
---|---|
ArgumentException | Child worker not found. |
KeepChildrenLevels
Gets or sets a value indicating whether to remove any children and descendant workers
after completion of the parent. This is useful when you need to access a property on
a descendant worker after the parent has completed, e.g. to check its
OutcomeStatus
or other property explicitly.
The setting value is inherited recursively from parent to all its child workers
when they start, but can also be set using AConfig with a
suitable ApplyTo
parameter.
Set to -1
to retain all descendants. 0
(the default for workers)
to remove descendants after the parent completes. A positive number to retain
descendant workers that many levels down.
The worker system gets a value of 1
by default, i.e. it will retain its
direct children when it completes (which allows checking properties of top level workers
after the worker system completes), but its grandchildren workers will by default be
removed on completion.
This property can be set on workers in the middle of the worker hierarchy, i.e. not just on the worker system.
Note: This property is thread-safe, and it can be set at any time, including after the worker has started running or completed.
Note that sibling and ancestor workers are always available.
Declaration
public int KeepChildrenLevels { get; set; }
Property Value
Type | Description |
---|---|
Int32 |
|
Locator
Gets the locator string, which represents where in the hierarchy this worker or
worker system resides. It consists of the names of all ancestors separated by
/
, e.g.: /Root/Extract Products/Read CSV Product File
.
The worker system at the top of the hierarchy defaults to the name "Root", but can also
be given a custom name when being created, which is useful when running different
worker systems together, e.g. during testing.
Note: This property is thread-safe.
Declaration
public string Locator { get; }
Property Value
Type | Description |
---|---|
String | The locator string. |
LogFactory
Gets the log factory for the worker system. It allows acquiring new loggers (if the preexisting ones on workers, ports etc. are not sufficient), flushing log messages etc.
Note: This property is thread-safe.
Declaration
public IALogFactory LogFactory { get; }
Property Value
Type | Description |
---|---|
IALogFactory |
Logger
Gets the logger instance for this worker or worker system, which should be used for
logging messages related to this worker system or worker.
The logger's name is set to the Locator
string of this worker or worker system,
which will appear in the logged output.
Note: This property is thread-safe.
Declaration
public ALog Logger { get; }
Property Value
Type | Description |
---|---|
ALog |
MaxRunningChildren
Gets or sets the maximum number of simultaneously running children allowed for this worker, or a negative number to not set a limit.
Note however that to avoid scheduling deadlocks, workers with input ports, i.e. transforms and targets, will be started irrespective of this setting. Transforms and targets still count towards the number of running children, and can therefore contribute to blocking workers without input ports from being started.
Defaults to -1, i.e. unlimited.
If greater than zero, no new children will be started until the number of running children is lower than this property.
If zero, no children will be started. Note that if no children are currently running, and no children can be started, the worker will have finished running its children.
As a more fine-grained alternative to setting this property on the parent, also consider setting IsStartable constraints on the children. See Worker Execution for more details.
Note: This property is thread-safe, and it can be set at any time, including after the worker or its children have started running.
Declaration
public int MaxRunningChildren { get; set; }
Property Value
Type | Description |
---|---|
Int32 |
See Also
Name
The name of the worker or worker system. All children of a parent worker must have unique names.
The worker system at the top of the hierarchy defaults to the name "Root", but can also be given a custom name when being created, which is useful when running different worker systems together, e.g. during testing. Other worker names default to their type name, followed by a number, e.g. "FileExistsWorker 1".
Note: This property is thread-safe.
Declaration
public string Name { get; }
Property Value
Type | Description |
---|---|
String |
RunningDuration
Gets the running duration - how long the worker has been running or did run.
Note: This property is thread-safe.
Declaration
public Duration? RunningDuration { get; }
Property Value
Type | Description |
---|---|
Nullable<Duration> | The nullable running duration.
Use |
Status
Gets the execution status of the WorkerParent
.
Note: This property is thread-safe.
Declaration
public WorkerParentStatus Status { get; }
Property Value
Type | Description |
---|---|
WorkerParentStatus |
WorkerSystem
Gets the worker system. Note that in the worker system instance, this will point to itself.
Note: This property is thread-safe.
Declaration
public WorkerSystem WorkerSystem { get; }
Property Value
Type | Description |
---|---|
WorkerSystem |
Methods
AddChildCompletedCallback(Action<WorkerBase>)
Adds a callback which will be called after each child worker that completes.
If the child worker is started, these callbacks will run, even if the child fails. If multiple callbacks have been added, they will all be called (even on failure) in the order they were added.
Each callback takes the worker that completed (typed as WorkerBase
) as a parameter.
These callbacks are useful for taking a specific action after each child worker completes, e.g. to track the progress of the child workers. The callback can of course check e.g. the name or type of the child worker to decide what action to take.
Alternatives to AddChildCompletedCallback()
include calling
AddCompletedCallback(Func<WorkerBase, OutcomeStatus, Task<OutcomeStatus>>)
on all or selected children, adding additional child workers with start constraints,
or calling RunChildrenAsync() explicitly and awaiting individual child workers.
Note: Adding the callback is thread-safe. The callback will later run on the parent's scheduling thread. The callback must be synchronous, and should return quickly to avoid delaying the scheduling of other worker children of this parent.
Declaration
public void AddChildCompletedCallback(Action<WorkerBase> childCompletedAction)
Parameters
Type | Name | Description |
---|---|---|
Action<WorkerBase> | childCompletedAction | The synchronous callback that will be called after each child worker that completes. See "Remarks" below for details. |
Remarks
Each callback takes the worker itself (typed as WorkerBase
) as a parameter.
If you need to access members of the derived worker type, there are two options:
- If you have access to the source of the worker, implement the callback as a (non-static) method in that worker. This allows accessing worker members without casting the worker parameter to the derived worker type.
- If you don't have access to the source of the worker, or you want to implement the callback using a lambda, cast the worker parameter to the actual derived type of the worker.
Examples
In this example we track how many child workers have failed:
AddChildCompletedCallback(w =>
{
if (w.IsFailed)
_numberOfChildrenFailed++;
});
Exceptions
Type | Condition |
---|---|
ArgumentNullException |
|
InvalidOperationException | Cannot add 'ChildCompleted' callback after the worker or worker system has completed. |
AddStartingChildrenCallback(Func<WorkerParent, Task<ProgressStatus>>)
Adds a callback which will be called every time just before the children run. These callbacks won't run if there are no children, or if the children won't run due to a AddStartingCallback(Func<WorkerBase, Task<ProgressStatus>>) callback.
When the callbacks do run, no further child workers or ports can be added, and ports cannot be linked (until optionally calling RemoveChildren()). Typical uses include:
- Configuring child workers and ports, even if they have been added externally. UnionAllTransform<TInputOutput> e.g. uses this to set port PortBufferingMode for its input ports, since it doesn't add them itself.
- Inspecting all child workers before they run
If multiple callbacks have been added, they will all be called (even on failure) in the order they were added, e.g. callbacks added in base classes will be called before callbacks added in derived classes.
The callback takes the worker or worker system itself as a parameter, typed as WorkerParent
.
Note: Adding the callback is thread-safe. Children won't be started until the callback completes, so ensure the callback finishes quickly.
Declaration
public void AddStartingChildrenCallback(Func<WorkerParent, Task<ProgressStatus>> startingChildrenFuncAsync)
Parameters
Type | Name | Description |
---|---|---|
Func<WorkerParent, Task<ProgressStatus>> | startingChildrenFuncAsync | The asynchronous callback that will be called every time before the children run. See "Remarks" below for details. |
Remarks
The callback takes the worker parent itself as a parameter. If you need to access members of the derived worker parent type, there are two options:
- If you have access to the source of the worker parent, implement the callback as a (non-static) method in that worker parent. This allows accessing worker parent members without casting the worker parent parameter to the derived worker parent type.
- If you don't have access to the source of the worker parent, or you want to implement the callback using a lambda, cast the worker parent parameter to the actual derived type of the worker parent.
The callback implementation is either synchronous and returns a
Task<ProgressStatus>
explicitly, or is async
and returns a
ProgressStatus
.
The callback should return:
ProgressStatus.NotCompletedTask
(orProgressStatus.NotCompleted
) to continue normal processing (which will run the child workers).ProgressStatus.SucceededTask
(orProgressStatus.Succeeded
) to neither run any child workers nor any AddChildCompletedCallback(Action<WorkerBase>) callbacks, and pass aSucceeded
status to any worker or worker systemAddCompletedCallback()
callbacks.- A failure
ProgressStatus
on failure, which will give the worker parent and anyAddCompletedCallback()
callbacks that failure status. Neither child workers nor anyAddChildCompletedCallback()
callbacks will run.
Examples
In this example we change any port BufferingMode
from Default
to Limited
,
which is e.g. what UnionAllTransform<TInputOutput> does, since it
doesn't require Full
buffering, despite having multiple input ports.
AddStartingChildrenCallback(w =>
{
foreach (var port in myWorker.InputPorts)
if (port.BufferingMode == PortBufferingMode.Default)
port.BufferingMode = PortBufferingMode.Limited;
return ProgressStatus.NotCompletedTask;
});
DisposeOnFinished<TDisposable>(TDisposable)
Adds the specified disposable to the list of objects to be disposed when either of the following occur, whichever comes first:
A: The worker (or WorkerSystem) completes after having run, or B: the worker never runs, and either the worker is removed from its parent, or the worker parent completes.
If either of the above has already taken place, the object will be disposed immediately.
For workers that have been started, the disposal will always happen after its
AddCompletedCallback(Func<WorkerBase, OutcomeStatus, Task<OutcomeStatus>>)
callbacks (if any) run,
and for the WorkerSystem
, just before Start()
/ StartAsync()
finishes. Note however that if a worker is never started, any
AddCompletedCallback()
callbacks won't be called.
Do not add any worker result property instance (e.g. Rows if it had been disposable) as a disposable, since that would dispose the object before any ancestor could retrieve the result.
An alternative is to use UsingActionWorker<TDisposable>. Also see Disposing Disposables.
Note: This method is thread-safe.
Declaration
public TDisposable DisposeOnFinished<TDisposable>(TDisposable disposable)
where TDisposable : IDisposable
Parameters
Type | Name | Description |
---|---|---|
TDisposable | disposable | The disposable object. Ignored if |
Returns
Type | Description |
---|---|
TDisposable | The specified disposable. This allows using or assigning the disposable directly, e.g.:
|
Type Parameters
Name | Description |
---|---|
TDisposable | The type of the disposable. |
Examples
Adb Keep Open Connection has examples of how to use this class.
GetDownstreamFactory<TInput>()
Gets the dataflow downstream factory for creating transforms and targets that are initially not linked to an upstream worker. This is only rarely needed, e.g. when a transform or target is created before their upstream worker has been created.
Note that in the returned factory, the UpstreamPort
will always be null
.
The newly created transform or target must therefore be linked
(using LinkTo(InputPort<TOutput>) or
LinkFrom(OutputPortBase<TInput>))
to an upstream worker before it and its siblings are run.
Also see Link, which is the more common way to create and link transforms and targets. See Worker Instantiation and Transform and Target Factory Methods for further details.
Note: This method is thread-safe.
Declaration
public DownstreamFactory<TInput> GetDownstreamFactory<TInput>()
where TInput : class
Returns
Type | Description |
---|---|
DownstreamFactory<TInput> | The dataflow downstream factory for creating transforms and targets. |
Type Parameters
Name | Description |
---|---|
TInput | The type of the rows for the "first" input of the transform or target to create. |
Examples
var target = parentWorker.GetDownstreamFactory<MyRow>().CollectionTarget();
RemoveChildren()
Removes all children. Must not be called when any child is running. Logs any errors.
This method is used by looping workers (in conjunction with RunChildrenAsync
that
add, run, and then remove children multiple times.
Declaration
public void RemoveChildren()
Exceptions
Type | Condition |
---|---|
InvalidOperationException | Children active (i.e. started but not completed) when removing children. |
RescheduleChildren()
Triggers the child worker scheduling loop, evaluating their IsStartable
property for workers to start.
Note: Only call this method if the default scheduling events are not sufficient, see IsStartable for further details.
Note: This method is thread-safe.
Declaration
public void RescheduleChildren()
RunAsync()
This method is called by the system during the worker or worker system Running phase. It typically contains all the worker or worker system specific logic, including initialization and cleanup. Override this method to implement any custom functionality required by the derived worker or worker system, in particular when deriving from many of the out-of-box workers or worker system with a "Base" suffix.
E.g., WorkerSystem uses it to implement the functionality of the Root(Action<WorkerSystem>) overloads, and SortTransform<TInputOutput> uses it to sort dataflow rows.
Note that if this method returns a Succeeded status and did not call a RunChildrenAsync() overload, then the system will run any child workers.
Also note that the Running
phase includes additional places where logic can
optionally be inserted via callbacks, to e.g. customize the initialization, cleanup,
and error handling of existing workers. This is mostly used when customizing workers
or worker systems that are not designed to be derived from (i.e. without
a "Base" suffix). See
Worker Life-cycle for details.
Declaration
protected abstract Task<OutcomeStatus> RunAsync()
Returns
Type | Description |
---|---|
Task<OutcomeStatus> | An OutcomeStatus describing the success or failure of the worker or worker system. |
RunChildrenAsync()
Run child workers asynchronously. They will not be automatically removed after they have finished running.
This method is used when the parent needs to await the running of the child workers, so that it can take action afterwards. This is used by e.g. looping workers that add, run, and then remove children multiple times.
Note that the caller should always check the return value, and normally return any
failure status as its own failure status. The system will however escalate any child
Fatal
status to the parent, if the caller doesn't do it.
Most workers will however not call RunChildrenAsync
themselves, in which case the
library will call it automatically once the worker RunAsync
method has completed.
Declaration
public Task<OutcomeStatus> RunChildrenAsync()
Returns
Type | Description |
---|---|
Task<OutcomeStatus> | A |
RunChildrenAsync(Boolean)
Run child workers asynchronously.
This method is used when the parent needs to await the running of the child workers, so that it can take action afterwards. This is used by e.g. looping workers that add, run, and then remove children multiple times.
Note that the caller must await
the completion of the returned task,
and should always check the result, and normally return any
failure status as its own failure status. The system will however escalate any child
Fatal
status to the parent, if the caller doesn't do it.
Most workers will however not call RunChildrenAsync
themselves, in which
case the library will call it automatically once the worker RunAsync
method and
any AddRanCallback(Func<WorkerBase, OutcomeStatus, WorkerParentChildrenState, Task<OutcomeStatus>>)
callbacks have completed.
Declaration
public Task<OutcomeStatus> RunChildrenAsync(bool removeChildren)
Parameters
Type | Name | Description |
---|---|---|
Boolean | removeChildren | If set to |
Returns
Type | Description |
---|---|
Task<OutcomeStatus> | A |
ToString()
Returns a String that represents this instance.
Note: This method is thread-safe.
Declaration
public override string ToString()
Returns
Type | Description |
---|---|
String |