My jaw dropped.
What happened at that very point was that the 20-30 minutes we'd already invested in the study became nullified. We could no longer ethically use this user as part of our study due to this "contamination in our specimen". What a pain.
When you ask observers to come in and help take notes and observe a user test session, it's very important that they have a clear understanding of what their expectations are. On the above mentioned test, I did verbally state that they must be silent, but for whatever reason simply forgot.
Was it that I wasn't clear in the way I spoke to them? Could they have been thinking about something I just said when I told them? Maybe that understood, but then forgot. Who knows. The problem to solve at this point is to help the observer become more aware, or to rank the information about their expectations at a higher severity level. To assist with this, I recommend that you establish some sort of "Observer Expectations" document that you can hand them and then walk through with them. This will help this more clearly understand what they are bringing to the table.
Here is an example of one such set of guidelines that may assist in this process...
Observer Expectation Guidelines
Show Up Early!
When you are invited to be an observer at a usability study, it's important that you show up early so that you can have an opportunity to get settled. Your attention to this detail communicates to the user that you are part of this cohesive and organized usability study team. It also communicates to the user that you respect their time.
If you feel that you have a conflicting meeting or that you must leave to go to some other prior engagement, it is better to decline an invite to be an observer. When you are in a usability session, you should count your time as belonging to the user. If you get up to leave early, you will likely become a distraction. It is also possible that leaving early will create an undue sense of urgency for the user. They may feel like they must hurry up so that they don't take up the usability team's time.
Be completely quiet!
You may not realize it, but you are normally really noisy. Grunting, laughing, body shifting, cell phone sounds, scratches, throat clearing, and other noises can be very distracting or can clue users. It's important to the sanctity of the test that you avoid contaminating the data captured for the study by being completely aware of these noises.
During the session, you must not talk unless you are asked, otherwise you are likely to cause any test findings to be terminated. The facilitator of the study will be looking for the user to struggle, therefore if you help the user then the purpose of the study has been nullified and everyone's time is wasted.
Don't ask leading questions
If the facilitator asks you if you have questions, be sure not to lead the user in answering the way you want them to. For example, "You probably really liked that feature didn't you?" is a leading question that should not be asked. A better way to find out what the person thought is to ask in this way, "What did you think of that feature? Did you like or dislike it?".
My New blog is http://thelucashall.com. Come see my newer/more relevant content.