Support Site

What is beta testing?
Beta testing is a stage of software development in which a new version of a software program is tested by real users in the real world using real data.

What is the purpose of beta testing?
Beta testing for software is very much like having a control group in a scientific experiment. It is possible to make discoveries in science without a control group, but without a control group it is very difficult to know if you've discovered what you think you've discovered. With software, you think you've developed a software program that meets the needs of the users, but without testing you can't know that for sure. Does it really do what the users want? Does it make sense to users? Do weird things happen under certain circumstances? Beta testing also tends to bring up suggestions for how to do things better; while it's often too late to include major revisions at the beta testing phase, these suggestions provide early ideas for what will be done in the versions to follow. These are the questions that need to be answered.

Who does beta testing? Why not the developer?
While the developer should and does do testing during development, it is extraordinarily difficult if not impossible for heavy testing to be done by the developer. The developer knows how it's supposed to work, so he can't look at it and determine if it makes sense to someone looking at it for the first time. The developer has only a limited number of computers to test with, and so he can't test a wide variety of systems. The developer is an advanced user, he cannot look at it through the eyes of a novice. During testing, the developer only has test data to work with, and so unexpected results often don't jump out as they do with the real data of a real user. Lastly, beta testers test entire systems as a whole the way the system makes sense to them; developers tend to test components individually and can't imagine all the variations of combining those components in all the ways users will do it. Thus, beta testing needs to include a broad range of regular users of all levels. Novices are as valuable as - if not more than - advanced users.

What sort of problems should be expected?
If we knew what to expect, beta testing wouldn't be needed. It really could be anything. Sometimes there are very serious, "obvious" errors because the computers and/or network used by the developer are "perfect" for the software, and once outside of that perfection display serious problems. Sometimes there are errors that only show up for particular users because of their own data or computer systems. Often everything works fine technically but "what I'm supposed to do" isn't obvious to users. Sometimes there are just typos or aesthetic issues. Once in a while, nothing is wrong at all.

How do I report a problem?
We will be using a discussion mailing list ("list serve") for beta testing so that other users are aware of issues already reported. So, any issues should be reported on the mailing list. Please feel free to ask questions; the more communication the better, and if you have a question chances are someone else has that very question themselves. When reporting the problem, please provide what type of computer you are using (Windows, Mac, Linux) as well as what browser you are using (Internet Explorer, Firefox, Safari, etc.). Please copy any error messages you get. Copy the URL of the page you are on so that I know exactly where you are in the process. Lastly - and this might be difficult - provide a step by step process for reproducing the problem. This last issue is often critical for solving the problem; reproducing the problem in the lab is often necessary to analyze where and what the problem is, and if it's not reproducable those steps often can't even get started. But please report even if you can't provide the steps to reproduce the problem; patterns of problems often emerge after many users report fleeting errors.

What do I get out of being a beta tester?
Besides the undying gratitude of the software developer, you have the opportunity to influence the software tools used by your fellow members of the Botanical Society of America, taking pride in the results of better software. If you have never paricipated in the software development process before, you will also gain insights and understanding of what is involved in the process.

Print | PDF