- Hits: 20550
A few weeks ago I sent an email explaining the JomSocial testing process and updating the status of JomSocial 3.0. We got some great responses from that newsletter and I wanted to give you another update.
First, you can be sure that I read ALL your comments, whether they’re placed on Facebook or written on our blog. I take them to heart. One of the first things I remember reading when I took over the JomSocial project was that some people felt that they "Paid to be QA testers" every time they upgraded. That’s when I realized our testing process wasn’t comprehensive enough to support an application this complex.
Since we hired QA expert Bryan Lamb to help us with a new testing process, he’s been training us, instilling new tools and practices, and helping us to implement this new process.
I never promised that this would be a fast process. It’s not. Testing right takes time, especially when you’re trying to put entirely new processes in place.
This is how the schedule looks:
Step 1: Creating the Test Cases - Completed
Duration — 2 weeks
Our goal was pretty enormous: to create 1,500 test cases. We ended up creating around 1,300. Enough to cover our bases.
Each test case takes time to write. The tester needs to:
- Take the steps;
- Monitor what happens
- Write the steps in our test case tool so that is understandable for anyone executing the test.
Here’s an example of one of our test cases, so that you can see what I mean.
The test cases were written by our support team. No one knows JomSocial better than them! We put all of them on the job and they finished it within two weeks. (They worked weekends too - thanks guys!).
The good news - we will never have to write these test cases again! We will be adding new test cases as we go, but we won't have to create a huge chunk of test cases again, we will simply execute the test cases already written on every release. This means that testing the next version will go much faster.
Step 2 - Testing Round #1 - Completed
Duration — 3 weeks
With enough test cases written to cover our bases, we started work on the testing. Once again, our awesome support guys were put to work. Each support person was assigned tests from different testers.
This really was a big job. Not only did we have 1,300 tests to get through, we had to test them all on four different platforms:
- Joomla 2.5
- Joomla 3.x
- Joomla 2.5 on mobile
- Joomla 3.x on mobile
That made for a total of 5,200 test cases — give or take a few tests that were irrelevant for mobile.
We found more than 100 bugs. That’s 100 bugs that you or your members won’t have to deal with.
Step 3 – Squishing the Bugs - Completed
Duration — 4 weeks (at the same time as testing)
Obviously, finding bugs was only the start. We had to squash them too. That was the development team’s job. While the support team was picking out the bugs, the developers were squishing them. We even added another developer to help speed things up.
Step 4 - Round #2 of Testing and Bug Fixing
Expected duration — 2 weeks
In my last email, I said that I wanted to hire an external team to test JomSocial. It’s become clear though, that hiring an outside team wouldn’t be the best approach. We realized that the testers have to be really familiar with JomSocial. Only people who really know the application can see the bugs because they’re the only ones who know how it should work. So we decided to keep the testing internal and run another round of tests instead. Each tester did different tests this time to keep the integrity high.
Step 5 - Beta Testing… and More Bug Fixes
If you’re wondering why we still need beta testing after two rounds of testing and debugging, the answer is that there are some things that we just can’t test ourselves. It’s only when the application is released into the wild that we can see how it works with different platforms, extensions and templates, etc.
How long will that take? It depends on how many bugs are found and how big they are. Hopefully after 2 rounds of testing and bug fixing, the time will be minimal.
After seeing the number of bugs that were found, it’s clear that this new testing process works very well. It might not be a quick process but it works. We’re digging out bugs that we wouldn’t have been able to find otherwise and we’re getting you the stable JomSocial that you need and deserve.
I know that you’re anxious to get your hands on the new JomSocial. I understand that completely. I hope though that you can see the value of a truly through testing process as much as I do and that you will bear with us a little longer while we add the last polish to the application.
New JomSocial 3.0 Demo Now Available
If you're curious about the new features and look, you can visit our new demo site, powered by Joomla 3.1 with JomSocial 3.0 Alpha installed. This demo rebuilds every 30 minutes.
- Front-End: http://demo3.jomsocial.com/
- Back-End: http://demo3.jomsocial.com/administrator (username: demo, password: demo)
Thank you for your continual support!
Questions or comments? Please post them below.