Persistence Files

After a QWFIX enabled process is enabled, a collection of persistence files will be created in the log directory.

If the process is killed and restarted again, QWFIX will automatically recover from the persistence files to the last state so that there will be no information loss. Every FIX message will be recovered by the combination of QWFIX recovery and FIX session recovery. Both can be performed automatically with very little burden (or even no burden at all) on developers.

 

Persistence Files

Below is a list of persistence files created by the RandomOrderManServer process.

20090428.Engine.Tutorial_OrderMan_Server.AllMsg.log
20090428.Engine.Tutorial_OrderMan_Server.InMsg.log
20090428.Engine.Tutorial_OrderMan_Server.SPI.log.dat
20090428.Engine.Tutorial_OrderMan_Server.SPI.log.idx
20090428.Engine.Tutorial_OrderMan_Server.StateInfo
20090428.EngineAlert.Tutorial_OrderMan_Server.idx
20090428.EngineAlert.Tutorial_OrderMan_Server.msg
20090428.EngineAlert.Tutorial_OrderMan_Server.tdx
20090428.EngineAlert.Tutorial_OrderMan_Server.typ
20090428.Session.Tutorial_OrderMan_41.A.adm
20090428.Session.Tutorial_OrderMan_41.A.in
20090428.Session.Tutorial_OrderMan_41.A.msg.dat
20090428.Session.Tutorial_OrderMan_41.A.msg.idx
20090428.Session.Tutorial_OrderMan_41.A.ouq
20090428.Session.Tutorial_OrderMan_41.A.out
20090428.Session.Tutorial_OrderMan_41.A.raw.dat
20090428.Session.Tutorial_OrderMan_41.A.raw.idx
20090428.Session.Tutorial_OrderMan_42.A.adm
20090428.Session.Tutorial_OrderMan_42.A.in
20090428.Session.Tutorial_OrderMan_42.A.msg.dat
20090428.Session.Tutorial_OrderMan_42.A.msg.idx
20090428.Session.Tutorial_OrderMan_42.A.ouq
20090428.Session.Tutorial_OrderMan_42.A.out
20090428.Session.Tutorial_OrderMan_42.A.raw.dat
20090428.Session.Tutorial_OrderMan_42.A.raw.idx
20090428.Session.Tutorial_OrderMan_43.A.adm
20090428.Session.Tutorial_OrderMan_43.A.in
20090428.Session.Tutorial_OrderMan_43.A.msg.dat
20090428.Session.Tutorial_OrderMan_43.A.msg.idx
20090428.Session.Tutorial_OrderMan_43.A.ouq
20090428.Session.Tutorial_OrderMan_43.A.out
20090428.Session.Tutorial_OrderMan_43.A.raw.dat
20090428.Session.Tutorial_OrderMan_43.A.raw.idx
20090428.Session.Tutorial_OrderMan_44.A.adm
20090428.Session.Tutorial_OrderMan_44.A.in
20090428.Session.Tutorial_OrderMan_44.A.msg.dat
20090428.Session.Tutorial_OrderMan_44.A.msg.idx
20090428.Session.Tutorial_OrderMan_44.A.ouq
20090428.Session.Tutorial_OrderMan_44.A.out
20090428.Session.Tutorial_OrderMan_44.A.raw.dat
20090428.Session.Tutorial_OrderMan_44.A.raw.idx
20090428.Session.Tutorial_OrderMan_Trader.A.adm
20090428.Session.Tutorial_OrderMan_Trader.A.in
20090428.Session.Tutorial_OrderMan_Trader.A.msg.dat
20090428.Session.Tutorial_OrderMan_Trader.A.msg.idx
20090428.Session.Tutorial_OrderMan_Trader.A.ouq
20090428.Session.Tutorial_OrderMan_Trader.A.out
20090428.Session.Tutorial_OrderMan_Trader.A.raw.dat
20090428.Session.Tutorial_OrderMan_Trader.A.raw.idx
Active.Engine.Tutorial_OrderMan_Server        
        

 

General Format of File Names

Note the persistence files has a prefix of the UTC date on which the process is started. If the process can be scheduled more than once (depends on the scheduler), a postfix will be appended to the file name so that the uniqueness of each persistence is automatically guaranteed.

 

FIX Session Persistence

RandomOrderManServer process manages 5 FIX sessions. Each FIX session has 8 files. Below is lists all files related to the session "20090428.Session.Tutorial_OrderMan_41".

20090428.Session.Tutorial_OrderMan_41.A.adm
20090428.Session.Tutorial_OrderMan_41.A.in
20090428.Session.Tutorial_OrderMan_41.A.msg.dat
20090428.Session.Tutorial_OrderMan_41.A.msg.idx
20090428.Session.Tutorial_OrderMan_41.A.ouq
20090428.Session.Tutorial_OrderMan_41.A.out
20090428.Session.Tutorial_OrderMan_41.A.raw.dat
20090428.Session.Tutorial_OrderMan_41.A.raw.idx

Note, a postfix ".A" is appended to the session name to indicate that this session is an acceptor session.

 

FIX Engine Persistence

There are 10 files for each FIX engine (process).

File names start with "20090428.Engine.Tutorial_OrderMan_Server" are for FIX message log.

File names start with "20090428.Engine.Tutorial_OrderMan_Server" are event/alert log persistence.

20090428.Engine.Tutorial_OrderMan_Server.AllMsg.log
20090428.Engine.Tutorial_OrderMan_Server.InMsg.log
20090428.Engine.Tutorial_OrderMan_Server.SPI.log.dat
20090428.Engine.Tutorial_OrderMan_Server.SPI.log.idx
20090428.Engine.Tutorial_OrderMan_Server.StateInfo
20090428.EngineAlert.Tutorial_OrderMan_Server.idx
20090428.EngineAlert.Tutorial_OrderMan_Server.msg
20090428.EngineAlert.Tutorial_OrderMan_Server.tdx
20090428.EngineAlert.Tutorial_OrderMan_Server.typ
Active.Engine.Tutorial_OrderMan_Server  

Active Engine Persistence

File "Active.Engine.Tutorial_OrderMan_Server" is the persistence for the current active FIX engine. There should be at most one such file per FIX engine. During the start up recovery process the QWFIX run time will look for the file to retrieve the information about the current active FIX engine.

 

Performance

A QWFIX enabled process with a lot of FIX engines will create a huge amount of persistence files in the log directory. Don't be scared by the amount of the files. In fact the reason we don't merge them into fewer files is that the performance is better in this way. Modern file system should have no problem handling that amount of files.