FastSessionContext QA and Benchmark

The "TestFast" project in QWFIX SDK sample contains functions used for QA testing and benchmark of the FastSessionContext class.

FastSessionContext QA Test

The "TestFast" project in QWFIX SDK sample contains functions used to QA the FastSessionContext class.

Both "TestFastMsgCodec" and "TestFastMsgCodec2" methods are used to test the message encoding and decoding in FastSessionContext. The both work in similar ways as described below:

1. Create new instance of FastSessionContext with appropriate FAST templates
2. Create a FAST message, randomly fill the fields in message
3. Encode the message into a MemoryStream, then decode a message from the same MemoryStream
4. Compare the decoded with the original message, report any difference as error
5. Repeat steps 2 to 4

There are some differences between those two methods:

• "TestFastMsgCodec" uses the FAST schema "QA1"; while "TestFastMsgCodec2" uses schema "QA2"
•  "QA1" and "QA2" FAST schemas are similar to each other, with some differences:
  • Most initial values of instruction operators are set in "QA1"; "QA2" leaves as many initial values unset as possible
  • "QA1" and "QA2" are used to test different state transitions in FAST encoder and decoder

Testing Result:

QWFIX FAST framework passed all testing scenarios.

FastSessionContext Benchmark

"BenchmarkFastCodec" method is used to benchmark the performance of FastSessionContext.

The method uses the same method to randomly generate the FAST message using schema "QA1". It then encodes the message into a MemoryStream and decodes the message from the same MemoryStream.

The program measures the time used to perform the following operations on a certain number of FAST messages:

1. Time to randomly generate the message, encoded and decode it
2. Time to randomly generate the message and encode only
3. Time to randomly generate the message only

With the time measurement listed above, we can calculate the performance of encoding and deconding FAST messages.

We don't want to include the time of composing the message because:

1. The message generation uses random number generator, which has higher CPU cost
2. In reality, we can use dedicate threads to perform message encoding and decoding, message generation can be performed in other threads. After all, multi-core CPUs and servers are becoming cheaper and cheaper.

Testing Result:

On a 2.4 GHz Core 2 Duo machine, with our QA template, the machine achieved about 2.4 million fields/second for encoding and 2.7 million fields/second for decoding, within a single thread.

In reality, the performance should definitely be better because the fields generated in testing are random fields, which require more bits to encode and decode.

Users are welcome to use the same method to test your own FAST templates and publish the benchmark numbers.