Launch environment
To run task definitions in a server environment, you can use TransformationRunner.exe, which launches TaskManager and initiates the startup task (in other modes it only initiates, or only starts the manager), or alternatively launches the processing module task without the "casing" in the form of a task module task.
TransformationRunner
TaskModuleRun.exe offers the following possibilities:
- control of tasks in the table TASKS (starting a new one),
- running tasks by running a Task Manager instance,
- when running a task,
- only resuming already existing tasks.
Possible call arguments of an executable program are presented below (running without parameters - starts a short environment auto diagnostics):
TransformationRunner [Version: 1.0.3947.22801 (x86)].
Copyright (c) 2005-2010 VSoft SA. All rights reserved.
Use Ctrl-X to try to stop tasks.
Starts a task in the TaskManager process.
Usage: TransformationRunner task_name [BatchId=number] [Revision=number].
[[parameter1=value1]],...,[parameterN=valueN]]
Usage:TransformationRunner --resume [...].
Example:TransformationRunner START batchid=1 revision=1 date=20100122
Parameters:
task_name Name of the task to run according to the task hierarchy file.
batchid=number Id. Batch of the task to be started. Ignored case sensitive in the parameter name. Acceptable value 'max'. Default - new batch (max+1).
revision=number. Repetition number of the running task. Ignored case in the parameter name.
parameterN=valueN Additional parameters of the running task in the form key=value.
-d Completion of processing does not terminate the program.
--resume Enables automatic resumption. Can be the only parameter.
--noCleanup Disables the ability to automatically clean up old, aborted tasks.
--monitorOnly Enables task monitoring mode. The tasks are not processed.
You can use keyboard shortcuts during operation:
Ctrl+Shift+t, Ctrl+t - adjust logging level +/-.
Ctrl+x - attempt to stop processing cleanly - with a summary of times in the log, etc.
No parameters. I only run the setup diagnostics.
Checking configuration...
DB_TASKS [OK]
ASMW_EXTRACT [OK]
ASMW_SPRAWY [OK]
ASMW_WORKFLOW [OK]
There are two ways of aborting the program execution:
- Forcibly (kill)
This action results in leaving active tasks in the state RUNNING. After possible problems with tasks running slow are solved, a database may be restored to its original condition. Transformation may be resumed by calling TaskModuleRun -resume. Unfortunately, if a transformation package is the currently executed task, performance summary, including processing time (which might be very useful for finding the reason of slow execution), will not be logged.
- Pressing Ctr-X in the console window
The action will request an interruption of a transformation. Appropriate errors of "Transformation interruption" type and summaries of operation time of blocks from tasks "Transformation packages" will be logged. Tasks will have the status "ERROR" set.
TransformationRunner
Program TransformationRunner.exe enables running a package transformation process. The table below shows the example of calling without parameters.
TransformationRunner [Version: 2.0.0.0 (x86)].
Copyright (c) 2005-2010 VSoft SA. All rights reserved.
Launches the Mass Data Transformation package.
Usage: TransformationRunner pathDoPackage.transform [--arg[=value] | pkgArg=value | block.param=value ...].
Example:
TransformationRunner load.its Input.FilePath=data.csv DateFrom=2010-03-12
TransformationRunner --dumpParams > params.txt
TransformationRunner load.its --paramsFile=params.txt
Parameters:
--dumpParams dumps the package parameter settings file to STDOUT
--paramsFile=path loads the package parameter settings file
Note: command-line arguments override the arguments in the file.
You can use keyboard shortcuts during operation:
Ctrl+Shift+t, Ctrl+t - adjust logging level +/-
Ctrl+x - attempt to stop processing cleanly - with a summary of times in the log, etc.
The required parameter 'pathDoPackage.transform' is missing!
An average transformation package may need many invocation parameters values and block parameters to be run. In order to enable calling many package variants with the most of parameters set, they may be passed in a file.
Such file may have the following structure:
### Parameters file template
### a simple character swap mechanism can be used:
# Var=value
# Block.Param=prefix{Var}suffix
FILE_PREFIX=20100917_1214
# Arguments:
LINK = server.local/SYSTEM/WebServices/Service1.svc
SCN_ID2_NAME = DEPOSITS_2
SCN_ID3_NAME = CREDITS_2
# Blocks:
## DIALER_Errors
DIALER_Errors.FilePath = LogDialera{FILE_PREFIX}.csv;LOGS
## CPK
CPK.Environment = SYSTEM_TEST
## DialerFileVindication1
DialerFileVindication1.FileListPath = ;DATA_OUT
## DialerFileVindication2
DialerFileVindication2.FileListPath = ;DATA_OUT
In a control file, it is possible to use a simple string replacement mechanism where string reference appears as {Key} and values may come from:
- the keys defined earlier in the same file,
- TransformationRunner.exe invocation parameters,
- appSettings section of a configuration file.
Values of parameters which are paths to files or directories may have the form: PathFragment1;DIRECTORY_SYMBOL While running, such syntactic structure will be translated to the resulting path and DIRECTORY_SYMBOL will be loaded from the configuration file of the section appSettings and then properly combined with the rest of the path.
Example:
Looking at the entry in the control file TransformationRunner.exe above:
## DialerFileVindication1
DIALER_Errors.FilePath = DialerLog{FILE_PREFIX}.csv;LOGS
entering in the application configuration file:
<appSettings>
<add key="LOGS" value="C:\archITekt\DATA\LOGS" />
<appSettings>
a configuration parameter for the file Errors_DIALER will take the value:
C:\archITekt\DATA\LOGS\DialerLog20100917_1214.csv
Service runtime environment
To host services, a program called TransformationService.exe is used, which can be run in test console mode or as a service (after prior registration in the system, e.g. with the sc program). The program in the course of operation will sequentially:
- will initiate logging according to the configuration (e.g. to disk, to database, etc...),
- if it is part of the technical configuration environment - it will perform configuration adjustments and register itself as a processing node,
- will check its own configuration (directories, availability of databases),
- will optionally perform auto-diagnostics,
- will look for all packages from the indicated disk or version server subtree with names that match the specified pattern and are service packages,
- will check and compile the packages as elements of a common SOAP / REST / POX endpoint,
- depending on how it is called - it will terminate the operation or start the web service according to the
<serviceModel>configuration and the found packets and enter the listening mode.
In the console-mode application responds to the <ENTER> signal, which throws into the logs of individual packages the current state of the operation, identical to the one that appears when the package finishes, supplemented by the current stack traces of all working threads.
Such an action can also be triggered by delivering a pause signal to the windows service. This behavior is used for diagnosis in abnormal situations - the log data can be loaded into the project environment and analyzed graphically.
The program has the additional ability to automatically restart when changes are detected in the content of service package definitions. The frequency of checking for changes can be specified in the configuration.
- Another additional option is the ability to automatically deduplicate element-root names of individual input operations when multiple packages use the same (usually default) name.
- Such an action, although not recommended in production environments, allows for rapid prototyping of large sets of network services.
- Deduplication is done by tacking on successive suffixes _1, _2, _3, etc... until unique names are obtained.
- On production environments, when this feature is disabled, a duplicate root name will result in the inability to start hosting services. Otherwise, the system could lead to data loss.
- Details of the configuration of the described elements can be found in the XML configuration of the program, equipped with comments.
The rules for building service packets can be found in the chapters devoted to the Request and Response blocks.
Operations defined in packages are docked to the WCF contract indicated in the configuration during execution. This allows some of the operations to be coded in .NET.
Version verification - replacement
Each processing package saves inside the software version used to produce it. In addition, at the end of processing or when loading a new version of an item, such details of the file (version server item) as MD5 hash, modification date and size are logged. The same values are available in the package designer as the "Package Version" option. By comparing the log and designer records, you can quickly verify whether the expected package swap took place (the right time, environment, etc.) or not.