We determined that the biggest obstacle to the matching system was that there was no central controller to handle all of the matching. Each Spark Core in the system was trying to handle its own matching, but this created a lot of redundant and jumbled data in the cloud. As such, in our next iteration, we decided construct a handler to which all of the Spark Cores would report for matching. The handler alone would run the matching algorithm and deal with the matching data, and all it would report to the Spark Cores was whether or not they had been matched.
Is this a good/useful/informative piece of content to include in the project? Have your say!