CopyCode SnippetWhen it comes to complex system, design is everything. In this article we are going to discuss the innovation we made in the design process and how it affects the final product.
A very small change at the begining eventually changes the whole thing. In our design, it all starts with - a slightly different FIX engine.
Our FIX engine is only slightly different than the other FIX engines, from the design perspective of view.
Traditionally, what a FIX engine does is "process and forget".
For example, if application wants to send a FIX message, it will do the following:
Similarily, when FIX engine receives a FIX message, it will do the following
Of course, that is what a FIX engine is supposed to do, by requirement. Our FIX engine does those tasks, too. Nevertheless, we also do something extra...
Appart from processing the FIX messages, our FIX engine does something extra.
It remembers every FIX message it sent or received. By doing so it maintains an aggressive in-memory cache of entire intra-day historical FIX communication. Our cache algorithm is very efficient. It can store 2-5 million FIX messages per gigabyte of memory. No matter what kind of business you are doing, we guarantee to find a way to cache your entire intra-day FIX messages in memory.
As a matter of fact, you typically don't have to cache every message in order to achieve optimal performance.
In our FIX engine, each intra-day FIX message is assigned a unique token. A token is 8 bytes long, equiverlent to a long integer. In fact, a token is a long integer in our Java version of QWFIX (QWFIX_J).
In our FIX engine, to send a FIX message:
CopyCode SnippetFIXMessage message = session.CreateMessage(msgType);
// Set the content of the message
FIXMessageToken token = session.SendMessage(message);
When we received a FIX message:
CopyCode Snippetpublic void OnMessageReceived(FIXMessage message, FIXEngineMessageHandlerStatus status) { FIXMessageToken token = message.MessageToken; // Other business logic handling }
The token can be used to retrieve a copy of the message in the future.
CopyCode SnippetFIXMessage message = FIXEngine.GetMessage(token);
Application is not recommended to keep a reference to the message. It is recommended to keep the token instead. Remember a token is a long integer?
Cost of retrieving a message by token is just a few CPU cycles, if the message in cached in memory.
This small extra feature turned our system into a - FIX message based Object Oriented in-memory database.
Nowadays, servers with 128 or 256 GB of memory are very common and very inexpensive. And it is always easy to split load of FIX based application to more than one machines. Also it's easy for QWFIX to guarantee every FIX message is cached in memory.
So what is the benefit? Well, it's big! The high level business logic is made extremely easy. Our QWFIX Order Manager is an example.
QWFIX Order Manager API is one example of how easy it is to implement high level business logic in QWFIX trading system.
From FIX version 4.0 to 4.3, the FIX protocol specification kept changing the meaning of fields in order and execution messages. That complicates a general purpose FIX version transparent order management system. We solved that problem in our order manager system.
The FIXOrder Object
In QWFIX Order Manager, an order is just a collection of message tokens. The tokens may be token of new order message, order modification message (cancel or cancel/replace), execution report message or cancel reject etc. We also keep an indices of the current order message and last execution message.
CopyCode Snippetpublic class FIXOrder { FIXMessageToken[] messages; int currentOrder; int lastExecution; }
The order object is very simple and compact because each token is just a long integer. It will be hosted in memory. And because each message has already been persisted by FIX engine, there is no need to persist order.
Helper Methods
QWFIX Order Manager provides helper methods to assist application creating FIX version specific order handling related FIX messages. For example, it takes one API call to create a "Cancel/Replace" FIX message of a specific order:
CopyCode SnippetFIXMessage replaceMessage = order.CreateOrderReplaceRequest();
The message will be created according to the specific version of FIX. Version specific tags will be added by the framework.
After the cancel/replace message is created, application only need to add business related tags such as price and quantity. And the same code can be used on sessions running any FIX protocol version!
Event Model
In order handling, applications not only just send FIX messages, but also receive FIX messages from the counterparties. The received messages will trigger corresponding events dispatched by framework.
For example, buy side applications may receive event notifications of execution reports and cancel rejects. Sell side applications may receive event notification of new order, order cancel or replace request etc.
Event model is also FIX version transparent.
Combine Them All Together
With this simple design, we now have a high performance, easy to use order management system that is FIX version transparent. With our order management API, application developers only need to write one set of code to handle the actural business logic, and the same code will work on all versions of FIX.
QWFIX makes implementation of any FIX based business logic extremely easy. FIX Order Management is just an example.
Another example is our standard QWFIX Order Routing Gateway. It is a general purpose order routing system. Unlike the similar systems from other vendors such as NYSE TransactTools, which operates on FIX message level and pretty much only does FIX message translation, our gateway operates on order level and understand every order in your system.
Other business logics such as pre-trade, post-trade, compliance and risk management can be easier than ever.
Visual Lambda is a GUI based progamming language. With Visual Lambda, users do programming primarily with a mouse, instead of using a keyboard.
We define business rules as signatures of functions. For example, a FIX message filter is a function that only takes one parameter, a FIX message.
We provide a GUI based Visual Lambda editor to help user build the rules (functions).
Visual Lambda is a great supplement to the QWFIX SDK. It greatly improved the flexibility of the entire system. With Visual Lambda, complicated business logics can be added later without modifying or recompiling code. It can even be "injected" into running process during runtime, compiled into machine code and run at full speed over there.
We integrated Visual Lambda with .Net, Java and C++ on all major OS platforms.
QWFIX trading system now comes with three flavors. QWFIX .Net is the pure Microsoft .Net solution. QWFIX_J is the pure Java solution. QWFIX_RTSJ is the pure Java solution with real time latency guaranteed for extremely high frequency traders.
QWFIX_C++ is the pure C++ solution, which is currently under development and will be released soon.
Every flavor of QWFIX follows the same design, very similar API and 100% interoperability. They share the same configuration and management tool and binary format of log files.
QWFIX is platform neutral. It supports all majoe OS platforms.
QWFIX configuration is done through our GUI based configuration tool, QWFIX Enterprise Manager. We complete eliminated the need for manually editing configuration files for FIX related configurations.
All configurations are stored into a centralized server called QWFIX Repository Sever. Our configuration system is actually a version control system for configurations. It automatically saves a copy of every change to configuration to a history store.
In another word, QWFIX Repository Server keeps the entire histroy of your configuration change. It is especially useful to operations such as history FIX log queries. In that case one may need the history configuration at a specific time.
QWFIX Enterprise Manager also provides graphical user interface for business logic handling configurations, such as Visual Lambda, order management and order routing gateway.
QWFIX offers a standard GUI based management and monitoring tool, QWRTAnalyzer. QWRTAnalyzer synchronizes entire intra-day FIX communication from selected process in real time.
QWRTAnalyzer performs remote synchronization with guarantee that there will be no performance panelty on the remote system. Synchronization is running in a thread with low priority and it only runs when more important jobs with higher priority are idle. Of course if the system is overloaded at 100% nothing is going to work. However on the other hand, if the system is overloaded at 100% it indicates there is a problem and remote synchronization is definitely not the cause of it.
Apart from the FIX message synchronization, QWRTAnalyzer also monitors the health of the remote process and performs other management functions such as runtime Visual Lambda business rules management.
QWRTAnalyzer understands a lot about your business logics automatically. For example, it understands your order flow and help you visualize and manage every single intra-day order in real time, no matter how many orders you have.
We also provide the real time remote management API, in pure Java or Pure .Net. For example, with our API, it is easy to build an application to remotely monitor the real time order flow of more than one remote processes. Developer only need to copy and paste the sample code we published. Our sample code gives application a collection of all live intra-day orders. A simple for-loop will run though the list and get every detailed information about every single order, at exceptionally high speed (everything can be in memory).
We also provide historical management API to inspect historical FIX logs. It provides the same interface to developer, with a collection of historical orders of more than one processes.
With QWFIX, management is easier than ever.
Different people has different requirement for the performance and quality of service. A retail broker may want a product with highest throughput, while a high frequency may want the system to response to any market event as quick as possible, preferably within guaranteed latency. QWFIX offers several different flavors for all kinds of customers.
For indepth information about the performance benchmark of different flavor of QWFIX system, please read our performance benchmark.
In our design, we always favor simplicity over anything. We won't sacrifice simplicity for a few percent of performance improvement. First of all, if the system is designed right, the performance should never be an issue and can always be taken for granted, which is the case in our design. Secondly, most of our customers don't even need to run our system on top of the line hardwares. Thirdly, the simpler we make it, the lower cost our customers will have to use it.
There have been numerous FIX engines, order management systems in the market right now. What is the benefit of using or switching to our system?
The practical reason is that, our system is extremely easy to use/develop on and it is extremely fast.
The philosophical reason is that, we follow a different design approach than the others. We believe it's all about information flow, and more importantly, the potential of information loss during the process.
Other systems use a FIX engine and SQL server configuration, or translate FIX message into some internal protocols. Some others use in memory database products from other vendors with FIX engine.
That is where we see problems, both information transmision and translation may result in information loss.
We are in the rapidly changing business of electronic trading. The regulation is changing consistently. You business may also change over time. For example if you are a sell side broker you may want to provide new products or services to your customer, which may result in new order types or even some new custom tags. If you are a buy side customer you may want to use those new products and services. Most of the changes will be reflected in your FIX protocol dictionary because most likely the FIX session is the only way to exchange information among different parties intra-day.
FIX message structure is a tree structure and it is extremely flexible. Because the rapidly changing nature of the business I described above, there is no way to know exactly what kind of information one is about to process during the design and development time of a trading system. The question is, how are you going to preserve information in your process? Unfortunately, if you use a database product of any kind from any other vendors, or theird party "middlewares". There is no way for you to handle this flexibility. Any change in regulation or business will most likely result in code change, and changes in database structure, and changes in other "middlewares". Cost will increase exponentially over time and the quality of the product will degrade over time.
Our system is the only one that can guarantee no information loss and "almost plug-and-play" straight-through-process from end to end. Using order management as an example. Application keeps a reference of every "FIXOrder" object. It can get any FIX message related to the order within a few CPU cycles. It can also inspect every piece of information, value of every tag within a few CPU cycles. It's extremely easy and simple, extremely fast and no information loss.
With our system, any change will only result in one modification to the FIX schema with our GUI base configuration tool (Enterprise Manager). We take care of the rest and you only need to concentrate to your high level business logics.
Also we have Visual Lambda, the most powerful business rule engine that can be proved to do whatever aother programming languages can do and is still extremely easy to use, which adds further flexibility to the entire system design.
Thank you for your time to read through our white paper. If you have any further question about our products and services, please feel free to contact us.