A summary of the events.
What happened ?[]
Memrise was running slow. Users experienced lagging and up-and-down performance for about 2 weeks.
Why?[]
- Memrise slowdowns were due to a massive amount of new user. Memrise was recently featured on the Wall Street Journal, BBC and Forbes. Catacademy brought new users indirecty which resulted in an enormous increase in traffic.
- What worked efficiently with small numbers of users became ineffective with large numbers of users. Memrise was growing quicker than its architecture has been designed to support.
Memrise had to :
- urgently upgrade
- re-structure so that the infrastructure could work at massive scale
- buid an infrastructure working at a scale 100x times greater
- reboot the database, use this delay to build an effective user experience.
What have they done ?[]
- Memrise did a lot of heavy server maintenance
- upgraded their database and systems and worked with Amazon (their host servers).
- performed processes with AWS (Amazon Web Service)
- added more servers
Safe Mode[]
- Every learner dived into Safe Mode, based on December 4th database to allow exam users to practise. Memrise said all data would be restored, but any data created during safe mode would not be.
- The Restoration took longer than expected
- The website was back at full speed after a huge database restoration.
Future[]
- The restoration of the database was designed so that it is now in much better shape to avoid any repeat scenarios.
- Usage capacity will be expanded progressively as much as they grow (servers will be added with AWS). They will scale the hardware and are continuing to work closely with the AWS engineers.
- Other steps are to come in the next few weeks : they will scale their userbase safely to much greater heights, and will make the whole site and associated mobile apps speedier.
The Memrise Team[]
They expressed their deepest apologies for this downtime, especially to all those who had final exams. They said they had in mind to be down only for a short time. It was a stressful experience for them.