Wednesday, September 24, 2014

TORParse Server Transfer/Updates

Here's some updates for you all - the transfer of TORParse to the new server is going along well. We've successfully copied the entire database to the new server along with all site files and the DNS records have been updated to now point to the new server (the [maintenance page]( you see now is being served from the new server). I'd like to give a big round of thanks to user "malloc" for his assistance in this.

What's holding us back from making the site go live on the new server is our continued conversion of the database from the "MyISAM" engine to the "Innodb" engine.

MyISAM is great for frequent reading, but performs poorly when you're writing data as frequently as we do for TORParse. To give you an idea, we had over 3 billion rows in our parse table alone - with that much data the hard disks couldn't keep up with the read/writes and you all probably noticed the slow performance and timeout result. Innodb allows for row locking instead of table locking, so we'll be able to run more efficient backups, our rank builder crons, multi processing on a single table, etc. without bottlenecking everyone else trying to use TORParse at the same time.

We estimate the conversion should take several more days at the least to complete, followed by some testing on our end to make sure everything is working properly with the new database engine. All logs previously uploaded should be intact when we go live again.

Down the line, we're going to start caching the top parses on the leaderboards so that they do not need to query the database every time someone wants to view them, which should significantly increase page load times as well.

The new server has 48 GB of RAM, a faster processor with more cores, and twice the HDD capacity as before, thus making it more expensive than the old one. We've been in the red the last four months, but I think that has more to do with the poor site performance we've been experiencing.

In terms of the client, we will be discontinuing any future updates for the Adobe AIR-based client, and instead will aim to create a new, better, open-source client, most likely as a .NET application. We'll create a Github or other source-control repository so approved community members like you can help contribute to it.

Thank you for your continued patience as we finish the server transfer process!

TORParse Owner

tl;dr - TORParse will be back soon, probably around October 1st, 2014, faster than ever before.

No comments:

Post a Comment