Some details on the main events of the past month.
Our main API servers received a Spectre and Meltdown update but after the reboot the performance was 10x worse. This cause notorious issues with the handling of the requests and we had to move the servers to new hardware. We changed some of the servers to a new provider in order to avoid having all the servers on the same location.
Note: the slow performance is not directly related with the patch. Other machines also received the update and didn’t suffer any impact.
- Our dashboard suffered a major impact during the migration issues that we had in the past month. During the migration described above we also moved our dashboard to new hardware and a new location. The 3h downtime was mainly due to the poor CPU performance after the patch.
Some of our machines are now running in Europe instead of NY. This won’t have any major impact on our final users.
Our monitor on the api.rethumb.com is hitting the CloudFlare cache. We will add new monitors with statistics from our own servers without the cache in front.