Everything that matters needs refining—a data collection instrument is no exception. I spent about a month designing the instrument (coming up with content that covered all the things we wanted to test) and testing in on my own. I probably spent another month just testing it with other people, which turned out to be hugely important. I only have maybe a handful of people go through the protocol, but each time I learned a lot and changed a lot. When I started out, I was having participants write things on index cards; that wasn’t effective. A lot of the tweaks were in the language: things that were obvious to me were not apparent at all to the participants, some of the instructions were not nearly specific enough, and some were too specific. Having other people see and use the instrument revealed a lot of things I never would have thought of and in the end that made the instrument stronger.

Leave a Reply

Your email address will not be published. Required fields are marked *