FRENDS Iron technical information

FRENDS Iron is a code/workflow execution engine with parametrization, scheduling, and monitoring/management interfaces. In Iron you may create schedulable Routines, which contain one or more Tasks which can be parametrized from Connection Points or Connection Point Groups.

Page contents

  1. FRENDS Iron elements - What are the base elements each Iron deployment comprises of
  2. Service Structure - What is FRENDS Iron made out of
  3. Development - How do I develop and deploy my own Tasks
  4. Routine parametrization - How can I parametrize my Routines
  5. Scheduling - How can I schedule my Routines
  6. Log service - How to log information from my processes
  7. FRENDS Cobalt - What can the FRENDS Cobalt file transfer engine do
  8. FRENDS Radon - A simple way to receive email alerts

FRENDS Iron elements

Routine

Routines are the main element which combines all of the other element types and are executed by the scheduler service.

Schedule

Schedules define when and how often a Routine should be executed. They can be configured in a versatile fashion allowing scheduling at a certain time, day of the week, day of month or months. Schedules may also be configured to be recurring with a custom interval.

Task

A Task is an executable element which may have configurable properties. A Routine may contain multiple Tasks which are executed in according to an execution graph, which may contain multiple branches which are split according to Task end states.

Connection Point

Connection Points can be referenced from a Task to provide values for configurable properties. In addition to containing public properties, encrypted properties may be used, which can only be read by the FRENDS Iron service.

Connection Point Group

Multiple Connection Points can be combined to a Connection Point Group, which may be referenced by a task instead of a single Connection Point. In this case tasks are executed in parallel for each of the Connection Points of the group.

FRENDS Iron service structure

The core of FRENDS Iron is a task scheduler and executor with parameter storage. The Tasks can be either custom .NET 4.0 code or Windows workflow Foundation 3.5 workflows. One or more of these tasks can be contained in a Routine with attached parametrization elements called Connection Points.

FRENDS Iron consists of:

  • Windows service that functions as a Task scheduler for Windows Workflow Foundation 3.5 and custom .NET 4.0 code
  • Parameter storage for Task execution
  • Microsoft Management Console for viewing and configuration of Task execution
  • Microsoft IIS web site for viewing and configuration of Task execution
  • Extensible Log service with Windows Azure Service Bus / Service Bus For Windows Server as a message bus
  • Extensible file transfers support out of the box with FRENDS Cobalt task
  • Email based templatable Windows Event Log reports for monitoring your solution 

Development

Developing tasks is straightforward:

  1. Create a class which has the instance method public void Execute() or public void Execute(CancellationToken cancelToken) and public properties as the parametrization options.
  2. Copy compiled code library containing class to the Iron installation directory.
  3. Create a task and routine in either the Web UI or MMC UI
  4. The routine and task combination is now ready to be executed and viewed.
     

Routine parametrization


A task in the routine may be parametrized directly, or through connection points. A connection point contains an XML data structure which can be referenced in a paremeter with the syntax {CP:xPath}.

 
A Task may have more than one connection point through Connection Point Groups. When the Task is configured with a group, multiple instances of the task are executed parallelly, each of which receives its parameters from a single Connection Point. 

Encrypted values are also supported with the {ECP:key} reference, the data is encrypted using AES with a 256-bit symmetric key. The key is encrypted with a 1024-bit asymmetric RSA encryption. The asymmetric private key is stored in the service configuration file encrypted with the Windows DPAPI and can only be decrypted by the Iron service user account. The public key is unencrypted and the UI can use it to encrypt the symmetric key.

Scheduling


Schedules can be configured with a wide array of options, e.g. they can be configured to trigger daily, on certain weekdays, or on a certain day of the month. The schedule may also have a limited execution season, e.g. the schedule should only be triggered between 01:00 and 02:00 AM or the schedule should not be triggered at all before or after a certain date.


In addition to the previous options, the schedule may have a list of calendar dates on which it should not execute, e.g. bank holidays. Once triggered, the schedule may be set to repeat on a given interval during the day, for example every minute.

Log service

The file transfers shown in the transfer view are logged through the log service, which offers an extensible way to log information from different sources and handle them in a centralized manner. The service can utilise Azure Service Bus or Service Bus for Windows Server for gathering log messages from different sources and processing them in dynamically loaded addins. We provide a log client library which handles the interaction with the service bus for easy usage, just provide the connection string and the message with some headers. We also provide the Cobalt log addin mentioned earlier and a bulk log addin which can write all messages to Azure storage for future processing with e.g. Azure HDInsight. Check out our blog post about the Log service for some more information.

Additionally the web UI offers a Transfers section specifically made for FRENDS Cobalt file transfers logged by the FRENDS Log Service. The section consists of two views, a transfer connections view, which shows all the endpoint systems and the Cobalt connections between them, and a transfer list view which lists individual transfer batches and their file transfers.

If you have bulk logging Message processing step enabled in your Cobalt transfer, you can download the full file to check if there was something wrong with its content.

FRENDS Cobalt

Cobalt is a file transfer engine implemented as an Iron task with straightforward but extensive configuration options. A Cobalt task configuration basically consists of three parts, source endpoint, message processing steps and destination endpoint. An endpoint is exactly what it sounds like, either the source or destination for files. The endpoint always has a type:

  • File - a regular file system or share
  • FTP(S) - a File Transfer Protocol server (with or without SSL)
  • SFTP - a Secure File Transfer Protocol server
  • Service bus - a Windows Azure Service Bus or Service Bus for Windows Server queue/topic/subscription
  • Custom - a custom endpoint class which is derived from our abstract EndPointBase class or one of the existing endpoint classes, for more information see the extension documentation

When transferring files between systems, there is often a need to do some sort of transformations or data gathering with the file, this is where message processing steps come in. Message processing steps are executed on the file during transit and as with endpoints, they always have a type:

  • PassThrough - Do nothing to the file
  • FileSystemBackup - Local backup for transferred files with rename pattern support
  • Xslt - Do an Extensible Stylesheet Language Transformations transformation on the file
  • Encoding - Convert the encoding of the file
  • BulkLog - Use the FRENDS Log Service's BulkLogger to store each transferred file in Windows Azure Storage
  • Custom - a custom message processing step class which is derived from our abstract MessageProcessingBase class, for more information see the extension documentation

FRENDS Radon

Radon is made for monitoring your environments, it read the event log and sends you an email if an event matching your filter has been found. Radon can be executed as an Iron task or a console application, which can be scheduled with the Windows scheduler. This way the reports do not need the Iron service to be running. The events can be filtered with a filter string which can match most of the event fields in a versatile fashion. For more information about filters, see the Radon manual. 

 

The sent email can use the default template or you can specify an email template to fully customize the message. The default template is a table with all the events that were matched by the filter.

FRENDS Iron technical information